Educational Leadership

November 2003 | Volume 61 | Number 3

The Challenges of Accountability Pages 6-10

Knowing how schools actually improve is our most urgent task, especially in light of the design flaws in No Child Left Behind.

Richard F. Elmore

In the past school year, I have visited about a dozen low-performing schools—schools that will probably be classified as failing under the provisions of No Child Left Behind (NCLB). I have visited classrooms and observed teachers teach. I have spoken to teachers and principals about their work. I have also spoken to system-level administrators about how they plan to respond to the accountability provisions of NCLB. I have learned something about the problems of low-performing schools and the processes required to improve them (Elmore, 2003).

As I look back on my experience, I am struck by the gap between what I observe in these schools and what NCLB says about accountability and failing schools. This gap is not surprising. One of the most robust findings of my 25 years of research in policy implementation is that policymakers usually know shockingly little about the problems for which they purport to make policy. In this instance, however, the degree of separation between the problems of failing schools and the policy prescriptions of NCLB is striking.

NCLB's Design Flaws

NCLB was an act of extraordinary political hubris. It is the single largest nationalization of education policy in the history of the United States, promoted paradoxically by a conservative administration with the docile cooperation of congressional liberals. Most of the normal institutional processes that precede the reauthorization of a major piece of federal policy got short-circuited prior to its enactment, so most of the expert advice on issues of testing, assessment, school improvement, and accountability that would usually have been brought to bear got ignored. The result is a law that has several profound design flaws (Elmore, 2002), a few of which are as follows:

Overinvestment in testing, under-investment in capacity building. NCLB aggravates a trend in state accountability policies. It focuses primarily on measuring growth in school performance against fixed standards—the so-called adequate yearly progress (AYP) requirement—and only incidentally on building the capacity of individual educators and schools to deliver high-quality instruction to students. This flaw is inherent to some degree in the politics of performance-based accountability. Standardized testing is relatively cheap and easy to implement. Capacity building is expensive and complex. Policymakers generally like solutions that are simple and cheap rather than those that are complex and expensive.

When we bear down on testing without the reciprocal supply of capacity, however, we exacerbate the problem that we are trying to fix. Schools search for short-term solutions—test preparation, for example—rather than longer-term, more powerful solutions, such as curriculum-focused professional development.

Ungrounded theories of improvement. NCLB judges a school's performance by the distance between its current performance level and the performance standard for which the school is being held accountable. The law requires more or less equal increments in growth—disaggregated by type of student—each year, a requirement that has no basis in empirical evidence about how schools improve their performance. The process of genuine improvement does not occur in equal annual increments. The AYP requirement, a completely arbitrary mathematical function grounded in no defensible knowledge or theory of school improvement, could, and probably will, result in penalizing and closing schools that are actually experts in school improvement.

Weak knowledge about how to turn around failing schools. The research on how to turn around failing schools is weak, as are the state and local policies and programs designed to address this problem. If one can draw any conclusion from that research, it is that a small number of schools may emerge from classification as failing schools, that some of these will quickly return to failing status, and that only a few will continue to improve after they have emerged from failing status. Many so-called "turnaround" schools are, in fact, functioning only at the minimal level required to keep them from returning to failing status. Turning around failing schools, in other words, is not the same as improving them (Ascher, Ikeda, & Fruchter, 1998; Viteritti & Kosar, 2001).

Perverse incentives for quality and performance. NCLB took a wide variety of state accountability systems developed in the traditional laboratory of federalism and converted them into a single design template, using federal funding as leverage. The law ironically introduces strong incentives for states that previously had relatively high standards to lower their standards and adopt lower-level assessments. If they don't lower their standards, states will be in the position of classifying far more schools as "failing" under NCLB than states and localities can actually handle. Most states are playing a delicate game of strategic interaction with the U.S. Department of Education around this problem. The accountability systems that preceded NCLB were diverse for good reasons, not least of which is that we are collectively at the low end of the learning curve around issues of school accountability. When you know less than you need to know to make intelligent policy, the most sensible strategy is to encourage experimentation and variability and to try to learn what works and what doesn't. NCLB has significantly narrowed opportunities for learning.

Policymaking by remote control. NCLB contains certain provisions—parental transfer rights from low-performing schools and stringent requirements for teacher quality, for example—that make sense in the rarefied ideological air inside the Beltway but make almost no sense to people who are trying to obey the law. Parental transfer rights, if they work, increase instability in enrollments in low-performing schools and adversely affect the distribution of students among schools, without necessarily improving instructional practice in either the sending or receiving schools. Requirements that teachers meet certain federally mandated quality standards—an idea that would have been considered absurdly interventionist not long ago—constrain the demand side of an extremely weak and idiosyncratic labor market, without doing anything about the supply side. Policy gets made in one place and implemented in another; how it gets implemented is someone else's problem.

Who Inherits NCLB's Problems?

NCLB is not the first example of poorly designed federal policy, nor will it be the last. The Title I law that it replaced, for example, had many similarly problematic features, including confusing performance requirements, underinvestment in capacity, and overreliance on testing.

What makes NCLB's design flaws so important is that they come with an unprecedented nationalization of education policy. This nationalization overrides the usual corrective processes whereby the 50 states moderate through adaptation the mistakes of federal policy.

Nor is there any likelihood that federal policymakers will revisit NCLB before these problems percolate through the state and local structures. Elected officials run on policy initiatives, not on their implementation, so as President Bush runs for office between now and November 2004, the debate will be about whether he has done anything about education, not about what he has done. In addition, Congress operates on a rigid reauthorization schedule; higher education and special education are ahead of elementary and secondary education in the queue. Congressional Democrats, who rolled over for the administration in the passage of NCLB, will not be eager to revisit their earlier lapses in judgment, even if they could. Democratic presidential candidates will avoid being drawn into the electoral tar pit of federal education policy, a classic "liberal" issue (where "liberal" is uttered with a slightly curled lip).

Chief state school officers, many of whom were very critical of NCLB when it passed, have undergone battlefield conversions, realizing that their objections to the law pale beside the necessity to keep federal money flowing by getting the U.S. Department of Education's approval of state plans. Governors, whose interest in the details of federal education policy is often tenuous, are busy handling their own fiscal crises. In other words, the likelihood that NCLB's problems will cause a political groundswell is next to nil.

Nor is the public likely to be sympathetic toward the problems of educators dealing with accountability systems. Most politically alert citizens, of whatever ideological stripe, work in organizations that have already internalized performance-based accountability. They find the complaints of educators about accountability to be out of touch and whiny.

In other words, the problems of NCLB will become the problems of superintendents, principals, and teachers—and eventually students and parents. Most people who have been in public education for more than a decade have experienced this process at least once. Policymakers generate credit for themselves, and for their bosses, by moving quickly from one issue to another within the dictates of two-, four-, and six-year electoral cycles, shipping their initiatives off to institutions that they understand only vaguely and episodically.

What Can Educators Do?

The best solution to the problems of NCLB, in the short term and the long term, is to focus state, local, and school resources and effort on the development of strong theories and practices of school improvement. In the face of policies like NCLB, professional educators have weak political authority and influence in part because they are fragmented professionally and lack strong cooperative theories on how to improve the enterprise.

I am shocked, for example, at the number of hand-wringing sessions in which superintendents gather with superintendents, principals with principals, and teachers with teachers. I am tempted to say that I will participate only when they all agree to sit in the same room at the same time and develop a common theory of improvement. When we first proposed a team-based professional development program at Harvard that included system-level administrators, principals, and teachers—the Harvard Institute for School Leadership—many believed that it wouldn't work because administrators wouldn't discuss their problems with principals and teachers from their districts, and vice versa. After 10-plus years, the program is successful, albeit still countercultural.

Here are some of the parameters of a strong theory of school improvement that grow out of my work on accountability and the problems of low-performing schools:

Internal accountability precedes external accountability. Educators are subject to draconian and dysfunctional external accountability policies largely because they have failed to develop strong and binding professional norms about what constitutes high-quality teaching practice and a supportive organizational environment. In our society, educators are usually people to whom things happen, not people who make things happen.

Not surprisingly, schools and school systems that do well under external accountability systems are those that have consensus on norms of instructional practice, strong internal assessments of student learning, and sturdy processes for monitoring instructional practice and for providing feedback to students, teachers, and administrators about the quality of their work. Internal coherence around instructional practice is a prerequisite for strong performance, whatever the requirements of the external accountability system. High internal agreement is the best defense against uninformed external pressure.

Improvement is a developmental process that proceeds in stages; it is not a linear process. Developmental processes—both individual and organizational—are not linear, but rather nonlinear, proceeding in stages. Performance-based accountability systems—especially NCLB—treat improvement as a linear process. Schools are required to make progress against a continually increasing gradient of performance. My work in low-performing schools—and in schools in general—has convinced me that schools increase their internal coherence and capacity around instruction in several discernible stages. These stages often involve significant gains in externally measured performance, followed by periods in which improvement in quality and capacity continue but improvement in performance slows or goes flat.

This phenomenon of gains in performance followed by flat spots that are in turn followed by gains makes perfect sense when you treat people in schools—adults and students alike—as learners. We do not learn in an easy, straightforward, incremental fashion, any more than we develop our cardiovascular capacity in a simple, linear way. We learn in part by tearing down old preconceptions, trying out new ideas and practices, and working hard to incorporate these new ideas and practices into our operating model of the world. It takes a while for these ideas and practices to "take," but when they do, they often result in learning at the individual and collective level.

Performance often lags behind practice. Schools are "improving" just as much when they are changing practices as they are when they are changing performance; performance, however, is easier to measure than is practice. Current theories of accountability do not allow us to make judgments about improvement that take account of the way improvement actually occurs.

Leadership is a cultural practice. The U.S. fetish for leadership leads to an overemphasis on the personal attributes of school leaders and a correspondingly weak focus on the technical, cognitive demands of instructional practice and the affective and behavioral responses to those demands. Successful leaders have an explicit theory of what good instructional practice looks like. They model their own learning and theories of learning in their work, work publicly on the improvement of their own practice, and engage others in powerful discourse about good instruction. These leaders understand that improving school performance requires transforming a fundamentally weak instructional core, and the culture that surrounds it, into a strong, explicit body of knowledge about powerful teaching and learning that is accessible to those who are willing to learn it.

Powerful leadership is distributed because the work of instructional improvement is distributed. Instructional improvement requires that people with multiple sources of expertise work in concert around a common problem; this distributed expertise leads to distributed leadership (Spillane, Halverson, & Diamond, 2001). Schools that are improving seldom, if ever, engage exclusively in role-based professional development—that is, professional learning in which people in different roles are segregated from one another. Instead, learning takes place across roles. Improving schools pay attention to who knows what and how that knowledge can strengthen the organization.

Knowledge is not necessarily where you think it is. One of the huge fallacies of performance-based accountability systems is the misconception that nominally low-performing schools don't know what they are doing and that nominally high-performing schools have something to teach them. This year, I have been in nominally low-performing schools that know far more than nearby nominally high-performing schools do about the processes of instructional improvement, creating settings with strong norms of practice, and managing the multiple demands of urban schools.

Most high-performing schools in our highly segregated society have gotten there not by knowing a great deal about instructional practice or improvement but by getting and holding on to students in high socioeconomic groups. The practice in most nominally high-performing schools is emphatically not about improvement but about maintenance of a certain level of confidence with the surrounding community. When I speak about improvement with people in these schools, they often look at me as if it had nothing to do with them. Most of the knowledge about improvement is in the schools where improvement is occurring, and most of those schools are, by definition, schools with a history of low performance.

An Urgent Need

The task of developing powerful theories of school improvement is urgent. The urgency stems in part from the difficulty of the work itself. Schools are low-performing in large part because their instructional practice and organization are not strong enough to meet the demands of educating children.

But the urgency also stems from the politics of education. Bad policy happens in part because of educators' weak knowledge, weak practice, and weak mobilization. We have deliberately chosen not to engage in powerful collaborative learning around the central problems of our work and have instead organized ourselves professionally and politically in fragmented ways. We have chosen to operate in ways that reinforce, rather than push against, the pathologies of the policies that affect our work. The discipline of developing a practice of improvement is one way to repair these problems.  

References

Ascher, C., Ikeda, K., & Fruchter, N. (1998). Schools on notice: Analysis of Schools Under Registration Review. New York: Institute for Education and Social Policy, New York University.

Elmore, R. F. (2002). The testing trap. Harvard Magazine, 105(1), 35–37.

Elmore, R. F. (2003, April). Doing the right thing, knowing the right thing to do: Low-performing schools and performance-based accountability. Paper presented to the National Governors Association Policy Education Advisors Institute, Los Angeles, CA.

Spillane, J., Halverson, R., & Diamond, J. (2001). Investigating school leadership practice: A distributed perspective. Educational Researcher, 30(3), 23–29.

Viteritti, J., & Kosar, K. (2001, July). The tip of the iceberg: SURR schools and academic failure in New York City (Civic Report No. 16). New York: Manhattan Institute for Policy Research.