Guest post by Michael Godfried: planner and policy analyst in Washington State.
Jerry Z. Muller’s The Tyranny of Metrics (2018) is a book that I believe Dr. Deming would have surely appreciated. This well-researched book gives an ever timely overview of the history and drivers behind the misuse of metrics that dominate organizational life in America and around the world. The bulk of Muller’s research is devoted to case studies in colleges, elementary schools, health care systems, policing, the military, business, finance, philanthropy and foreign aid. Muller concludes with a chapter on his proposed proper use of metrics. The Tyranny of Metrics is a wise, concise book that can be read enjoyably in a few sittings.
Muller is a history professor who has written books on Adam Smith and Capitalism. His work has appeared in The New York Times and the Wall Street Journal. As a history professor he provides fascinating context for the rise of metrics. As early at 1862, the English Parliament was proposing a ‘payment for results’ education reform. In America, the use of metrics can be tracked from Frederick Taylor’s Scientific Management (1911), the New Public Administration of the 1980s and the 2001 No Child Left Behind program and beyond. The loss of public trust in institutions in the 1960s and 70s was a driver in the heavier reliance on seemingly ‘objective’ metrics.
Muller talks about what he has experienced first hand. As head of a university department, he saw the impacts of the metrics arms race. He saw increasing time and resources channeled to generating and managing metrics and drained away from more worthwhile efforts such as course development and mentoring. Administration costs for many institutions have ballooned due to tracking and generating metrics.
Among the many startling examples from various fields, the one relating to the Vietnam War is perhaps the most chilling. Secretary of Defense Robert McNamara chose ‘body count’ as the primary index of American military success. This abstract metric was not supported by Generals in the field. They recognized other factors, within a larger strategic view, mattered much more. Under McNamara, the armed forces sought to meet productivity targets of bombing sorties, shells fired and body counts. As Muller comments: “What could be precisely measured tended to overshadow what was really important.”
Few people understood measurement as well as Dr. Deming. But he was also fond of saying that the most important information ‘is unknown and unknowable.’ He saw the terrible waste that occurred in organizations due to the misuse of measurement. Dr. Deming cared most about how measurement was appropriately used to better understand and improve the overall system. In The New Economics (TNE), Deming describes one misuse of metrics in the form of numerical goals:
“A numerical goal leads to distortion and faking, especially when the system is not capable of meeting the goal. Anybody will meet the quota (goal) allotted to him. He is not responsible for the losses so generated.” (TNE p31)
The following quote from Muller is a perfect complement to Dr. Deming’s quote:
“…(Charles) Goodhart’s Law, which states, “Any measure used for control is unreliable.” To put it another way, anything that can be measured and rewarded will be gamed.”
Muller does not mention Dr. Deming is his book but does cite Deming disciple Don Berwick and Deming collaborator Alfie Kohn. Common Deming themes of pay for performance, ranking and ratings and intrinsic motivation are also discussed in the book. A sense of a larger system is always present for Muller although not as directly stated as in Dr. Deming’s philosophy on management.
This slim book is surprisingly rich in insight and example. It may well become a classic. Certainly, The Tyranny of Metrics is worthy to be read as a companion to The New Economics and Out of the Crisis.
Mike shares an interesting story about being invited by Dr. Deming to present his ideas on the idea that “Management is an Analytic Problem” at a New York University Deming Seminar for Statisticians. In the video above, Mike shares some of the ideas he discussed at that seminar, including showing how several of the 14 points and 7 deadly diseases amount to stopping existing “enumerative practices” and beginning other analytic practices.
Mike shares a quote from W. Edwards Deming’s The New Economics (page 94*):
The system of profound knowledge provides a lens. It provides a new map of theory by which to understand and optimize the organizations that we work in, and thus to make a contribution to the whole country.
Mike shares a great story on his experience preparing to run a marathon:
I achieved my goal by not my aim. That happens a lot, we honestly translate aims to goals. And then we do stupid things in the name of the goal that gets it the way of the aim. We forget the aim sometimes and put the goal in its place.
In my personal opinion Mike’s Deming 101 presentation is exceptionally valuable and is packed with important ideas. I strongly recommend watching this presentation (and rewetting it if you last watched several years ago).
Beginning in 1951, the Japanese Union of Scientists and Engineers (JUSE) has awarded the Deming Prize to honor organizations, and later, individuals, for extraordinary accomplishments in quality management. Recipients receive a prestigious medal, complete with an image of Dr. Deming, with his quotation, “The right quality and uniformity are foundations of commerce, prosperity and peace.”
Students of Dr. Deming know of his appreciation of the statistical control of variation, a body of work pioneered by his mentor, Dr. Walter Shewhart. Guided by Shewhart, Deming learned the significance of managing variation, with appreciation of both common and special causes, and the economic value of stable processes, with predictable levels of variation.
It doesn’t follow that because Dr. Deming suggested efforts to identify (and remove) special causes of variation and sought to manage common cause variation (to achieve the “right uniformity”) that caused processes to be unreliable and that harmed customers, it meant he was against variety. Nor does it mean that he failed to understand the importance of variation in different contexts than the context where it caused problems. Drawing such a conclusion is just not sensible when looking at Deming’s larger body of work, extending from his employment at the Department of Agriculture to the publication of his last book, The New Economics. The conclusion that Dr. Deming was advocating the elimination of variation is a common misunderstanding that is usually caused by taking one quote and drawing conclusions about what that quote meant which are in conflict with his System of Profound Knowledge.
As evidenced by his theory of management, Dr. Deming understood the importance of seeing any organization as a system and how it understood variation fitting within that system, including the example of matching the seasonal variation in demand for an organization’s services with variation in the staffing of the organization.
This webcast shows Sarah Pavelka’s presentation, Deming and Lean: The Disparities and Similarities, at the 2012 Annual Deming Conference. Sarah co-authored Tool Time for Lean with David Langford. In a previous post I wrote about this presentation which included a few selected clips from the presentation. Now we have added the full presentation to our YouTube channel:
When I go out to help folks I say “what drives you nuts?” What do you do that you would like to skip? Individuals are really good at creating band-aids to processes. If they don’t like the process they will create a workaround. I wasn’t to hear about those workarounds… They may not understand why [that thing they skipped] was important… Or have they learned it is not important.
This is one good tactic to use when introducing new management ideas. One important aspect of the initial effort is to get people to see the change as something positive, something that will make their life better in addition to improving value to customers. Taking care to pay attention to what they want changed both shows an understanding of psychology (“respect for people” in lean terms) and is likely to unearth some fairly easy improvements to start with.
This effort also can be an opportunity to improve communication and the understanding of the organization as a system. If there are some cases where the short-cuts did not just eliminate non-value added steps but eliminated steps that were important even though it wasn’t obvious to those doing the work (and choosing to skip those steps) giving people an understanding of why what they do matters makes a big difference to them.
Leaders must understand the system… if they go back to that flavor of the month they are going to struggle and they are definitely going to struggle with Deming or the lean concepts.
In this presentation Sarah discusses the difference between criticism of lean based on what is promoted by wise lean thinking leaders and criticisms of the poor implementations of lean. Often I see criticism of “lean” made by criticizing poor management that labels itself “lean” but does little that justifies such a claim. I don’t see much value in such criticisms. There is value in looking at what differences in emphasis, tactics and principles there are between Deming and lean efforts. In the previous post I discussed that idea in more detail.
Don’t shut the door on other continuous improvement philosophies.
I agree. I see plenty of those interested in Deming’s ideas that would improve their ability to manage with more appreciation for other management improvement efforts. I do believe that creating a management system based on Deming’s ideas is the best strategy to pursue. But that doesn’t mean there are not good management practices at organizations not following Dr. Deming’s System of Profound Knowledge that can be learned from by those interested in Deming’s ideas. W. Edwards Deming himself would certainly be learning what he could from others, as he showed through his behavior his entire life.
2015 ASA Deming Lecture by William Meeker – Reliability: The Other Dimension of Quality:
William Meeker begins the presentations discussing his experience with W. Edwards Deming in relation to a paper by Meeker and Gerald Hahn. The core issue related to the issues around problems in how analytic studies are treated. We have a previous post on this blog on this topic: Enumerative and Analytic Studies.
William also mentions Dr. Deming’s paper, On Probability As a Basis for Action, which we make available through our website. William summarizes what he took from the paper in reference to analytic studies:
Enumerative study: Making inferences about a static population after sampling from a corresponding frame. Analytic study: Everything else and especially sampling from a process.
Dr. Meeker showed a letter Gerald Hahn received from Dr. Deming on their paper:
Your paper is great. You refer to a new book that you published. I should be pleased to have a copy. To go on, your paper pleases me very much. In the first place, almost nobody seems to be interested in the difference between enumerative studies and statistical inference, which I called analytic problems. It pleases me much to see not only your interest in the distinction but great contribution thereto.
Dr. Deming subsequently invited Meeker and Hahn to present their paper at an upcoming Deming seminar for statisticians. During their presentation there was a great deal of vigorous debating – much of it from Dr. Deming himself. I think this is an example of some of Dr. Deming’s strengths: his lifelong interest in continually learning (you can follow his lead and read the paper he spoke so highly of, which I linked to earlier in this post) and a willingness to explore new ideas even while challenging some of the ideas being presented.
Dr. Deming said your sources of power come from primarily 3 sources. The first source is your formal position, as a student you don’t have a lot of formal position, you can’t just walk into a professor’s class and just say you’re really screwed up lets change this – just not going to work out well for you. The second source of power is knowledge. The third source of power is personality.
You may have worked with people in the past, very knowledgable people, I mean extremely brilliant people [but] have no personality. You can’t get anything done because you can’t work with them. Or you may have worked with people in the past and he is just a great guy to be around or she is just a fun person to be around and everything else but they are just clueless, they have no knowledge. Or you have someone in a formal position but they have no knowledge or personality, they have a real problem there.
Deming often said, hey you want to affect change, don’t use your formal position: that is the least effective methodology for changing the situation.
We have this concept we have to motivate people… when you understand the brain research and understand the psychological research you understand that you can’t motivate anybody. We are already well motivated…
The only thing you can do as a manager is demotivate people. When you start thinking like that you start thinking how am I operating, what am I doing that is causing people around me to be demotivated? Am I preventing them from thinking?
I believe that a manager can also take action to reduce the systemic demotivation present in the workplace (and I believe David agrees). There is a big shift in thinking from thinking that a manager must motivate people to thinking a manager needs to remove the barriers to people’s intrinsic motivation (this of course was explained by Douglas McGregor in 1960 with theory x and theory y thinking in his book The Human Side of Enterprise).
TJ Gokcen,CEO of Acquate, shared a presentation at our 2015 International Deming Research Seminar on Aim as a System.
In the presentation TJ says that one of management’s responsibilities is to coordinate communication between the interconnected components of a system.
Ensuring that feedback is shared with the parts of the system that need that information is something that sounds obvious and may not seem complicated but it is a challenge that many organizations have great difficulty with.
The effort to make this happen adds costs to the organization. Determining which costs are worth taking on and which (while they potentially may be worthwhile) should be avoided (because the benefit is not worth the cost) is a challenge.
The aim of the system sometimes may require management to change the boundaries of the components and therefore redefine them. And this is important because every system has a boundary and you determine these boundaries. These boundaries are not natural, actually management defines them.
It is also [of] utmost importance not to wait until a crisis or problem arises or merely react to circumstances to define the boundaries of a subsystem.
Often we resort to looking at the organization chart to “define” boundaries (really it isn’t defining boundaries, rather it is just defaulting decisions and blame to parts of the hierarchy without understanding the systems involved). This is easy but it is not a wise way to manage when you understand system thinking concepts and instead view the organization as a system.
The video shows the presentation by Christine Simpson and Sarah Ambrus at the 2015 Deming research seminar: Empowering Students to Lead Change. They gave a similar, and a bit longer, presentation at the 2015 Deming in Education conference on Student Led Change.
Quoting a student:
Would you ask a dentist to fix your car.
Students at the school are taught critical thinking skills while learning to apply quality tools to improve the system of education in Leander schools. That creates a culture where thinking systemically is natural (for not just teachers but students as well).
Start somewhere. Start somewhere with continuos improvement. Start somewhere… It is ok if you don’t have everybody on board, your acts and your progress will take on a life of its own and you will gain supporters and you will transform your school. It will happen.
The video shows the presentation by Ron Moen, Prediction is the Problem, at our 2012 annual conference. A previous post on our blog in 2013 included a clip from this talk and explored Ron’s thoughts which might be of interest if you enjoy this presentation.
You may benefit from reading my previous post on enumerative and analytic studies before you watch the video (if you don’t clearly understand the importance of knowing when to use analytic thinking).
use statistics to support the learning of subject matter knowledge. That was a breakthrough in the book. We were always trying to use statistics to enhance the subject matter knowledge. That was the key.
This idea seems simple and maybe unimportant but if you question how data is used in your organization you will often find it separated from subject matter knowledge. When you try to act on data alone, rather that use data to enhance your deep understanding of the specific processes and systems in your organization you can quickly misuse statistics to lead you astray.
Keep the data in its rawest form… Plot the data in the order it was generated… Make changes and use the data to decide if the change was an improvement. Any sort of aggregation and sort of symmetric function, summary statistics, you lose the power of the information.
One challenge is to create a culture that expects data to be used to improve learning and decision making. However, the use of data is not sufficient. The data must be used properly and this point is much more frequently an issue than I would hope. It is important to create systems that not only encourage the use of data but do so in a way that avoids the problems so often seen without an understanding of variation, or the difference between analytic and enumerative data, or without an understanding of other risks to misuse data.
2017 ASA Deming Lecture, W. Edwards Deming – A Kaizen Statistician, by Fritz Scheuren, NORC-University of Chicago. In his presentation, Fritz provides a personal view of W. Edwards Deming the man and Deming’s ideas.
That’s the way people understood Deming. As a critic, not as a person that gave you the opportunity to improve. That’s a problem we still have, not as much as we did then though.
Such an attitude provides an easy escape for those executives that don’t want to change. Instead of accepting the challenge to improve, just position the ideas as criticism without suggested steps to improve. Then it is easy to keep doing what you have always done.
Obviously, Fritz is not saying that everyone held such a view. The many examples of organizations adopting Deming’s ideas to improve illustrate that not everyone did; but Fritz does provide an explanation for how some people miss the opportunity to improve when they dismiss Deming’s ideas only as criticism.