This summer I worked for the non-profit organization The New Teacher Project (TNTP), which is in the middle of a five-year contract with the Houston Independent School District to reform the human capital system of HISD. The changes happening at HISD quietly place the district in the center of the most controversial debates in education policy – the use of value-added modeling to evaluate teachers and paying teachers in part based on how well their students perform.
After studying the theories and reading all sides of the debate surrounding education policy, I hoped that working in a reform-minded district like HISD would help me settle my internal conflicts about education policy reform and come to a conclusion about the benefits and harms of these policies. I’m still searching for that answer, but I did learn some important general lessons about the difficulties of policy implementation.
How to implement policy is the $64,000 question. Most of what we learn at LBJ falls into the buckets of policy analysis and policy development. But the truth is, at some point policy analysts must hand off their policies to a group of people to implement. This is a crucial stage and one I expect most policy graduate students are unprepared for.
A policy analyst would like to think that a smart, well-designed policy will be implemented smoothly and faithfully. But even small changes are difficult to implement successfully. Very few people like change and most institutions value stability over the ability to make changes quickly. The ability to get buy-in from stakeholders and shift their mindsets is both crucial and extremely difficult. (Think about what it would take for you to change a habit. Now multiply that by the size of your organization.)
Most policy reforms look something like what already exists, and without a cultural change in the organization, it is too easy for existing behaviors to adapt to new procedures. Hypothetically, if a teacher evaluation system rates 95% of teachers “effective” versus “ineffective,” and the district administration decides to adopt a more differentiated system rating teachers from 1 to 4, without a mindset change, we would expect that 95% of teachers are now rated a 3 or a 4. Was the policy reform successful?
Research matters. Except when it doesn’t. The accumulation of knowledge that we call “research” can be used and misused in many ways. Good researchers and consumers of research are careful to qualify findings and consider questions of generalizability. However, in reality, research often functions to bolster agendas and is sometimes selectively ignored.
Research can impact policies that affect people’s lives, but nobody makes decisions based solely on the soundness of the research. Instead, it is one of many factors that affect decision-making. For example, class sizes in the U.S. have fallen steadily over the years, partly because of research that showed a smaller student-teacher ratio could have a positive effect on student outcomes. Now that the cost of this policy has become higher than citizens and governments have decided they want, class sizes are rising again.
The current focus on teacher quality (somewhat reflexively defined as a teacher’s impact on assessments) stems from research that shows teachers are the most important in-school factor affecting student outcomes. Combine this finding with another research finding that, in general, the factors that determine most teachers’ salaries do not correlate with student achievement gains. Add increasingly restrictive budget limitations and these two factors lead to the growth in movements to tie teachers’ compensation to student outcomes.
This is an instance when research matters. But what about the research that shows pay-for-performance schemes usually do not impact student achievement? For some reason, that research doesn’t matter.
Being detail-oriented is overrated. How many of us have described ourselves as “detail-oriented” to a potential employer? Although this is a valuable skill, one that is in even shorter supply is being “big-picture” oriented.
This skill is actually something that I think LBJ prepares us to do, but I didn’t fully appreciate how valuable this ability was until this summer. I was fortunate to interact with many hard-working and talented teachers, principals and district administrators. But the passion and urgency they brought to their work sometimes hindered their ability to see beyond their own domain.
Teachers who are devoted to their classrooms and students found it difficult to empathize with the difficult decisions a principal may need to make for a school. Principals immersed in the multiple duties required to lead a school could not look outside their walls and consider the impossible tradeoffs that district administrators face. Even working primarily with the Human Resources Department, I forgot that others in HISD were worrying about the $1.9 billion bond or the Title IX complaint.
Unfortunately, I wasn’t able to debate the merits of value-added modeling with Superintendent Grier or run a linear regression (not one!) this summer. What I did learn was that our education at LBJ is truly just the beginning of a lifelong education in policy.