Another principle of improvement science is that we can not improve at scale what we can not measure. Mickey Corso, of the Quaglia Institute, explains the principle by likening it to another use of scale, losing weight. If I make a decision to lose 15 pounds in the next three months I will need to implement one or more changes to my behavior such as eating a particular diet, reducing sugar or fat intake, walking more, or exercising more days a week.
Let’s take one of these change ideas, walking more. The premise is the more I walk, the more weight I will lose, so I need to know if I am walking more. You likely have seen the commercials for or may even have the gear that records how many steps you have taken. Each day, with this pedometer, I can track how many steps I have taken throughout the day and if more is needed as the day draws to an end. I am able to chart this data each day and over the course of days, I will know how well I have executed my change idea of walking more.
Before I even step on the scale to see if there has been an improvement in weight, I am able to measure how well I am implementing a change in practice. In addition to charting the quantitative data (number of steps), we also record the qualitative data here of the process. When was the best time to walk – morning, afternoon, or evening? When were we able to get in extra steps in our regular routine? When were we most motivated? When were the walks most enjoyable? When were they most difficult?
After a period of time (one week, one month, etc.) I get on the scale to see if the change in practice has made any difference in my weight. My weight and the delta in weight since the last time I stepped on the scale is my outcome data. More importantly, since I have measured the process I am able to learn more about how the change idea has affected improvement. Of course, it is important to keep other practices and behaviors the same during this time. If I am walking more and eating more, I will not arrive at the same understanding. Or if I am walking more and eating less, it will be difficult to isolate which change idea has had more impact.
In his book Tony Bryk distinguishes between these two different types of data:
- Measurements for Improvement determines whether a new practice has formative value by measuring work processes that signal actionable changes (ie. walking more).
- Measurements for Accountability are summative, global measures of performance outcomes applicable to all that identify exemplary or problematic individual teachers, schools or districts (ie. weight).
Again, it is important to remember that the improvement process does not need to mean a complete change in behavior and practice – eating less, eating better, walking more, and exercising more – which may be more difficult to maintain over the long-term. Rather, we can introduce one change idea in one area of practice with one person, learn from that process, and then apply it at a greater scale if we were seeing the intended results.
For school improvement, we apply a similar mindset. Measurements for Improvement keep us focused on a school improvement process driven through smaller, isolated chunks rather than large, full-blown, multi-component initiatives. This requires patience over several year and given our sense of urgency in urban education, this is a difficult discipline. Our tendency is to throw every change idea we have at the problem right from the start. If the problem is low literacy skills, we introduce a multifaceted initiative with intensive training on new curriculum, instruction and assessment that requires a complete overhaul in practice for teachers. Those who have attempted a similar approach to weight loss know that you may be excited by positive short-term changes and then become disappointed by long-term results because such drastic change in behaviors is not sustainable.
By implementing smaller, more manageable change ideas that are more realistic in demands on leaders, teachers, staff, students and families we can learn more about how each process works and the impact it has had. If it yields the results we were looking for, we can implement it at greater scale (more often in more classrooms with more students and teachers, etc.) the next year.