Dr. David Wetzel — INCOSE

A common image of metrics on posters and the internet is of a triangle with “Good,” “Fast,” and “Cheap” on each side and the saying “Pick Two” in the middle. The implication is that if you pursue any two of them, then the third one suffers. For example, if you want good and fast, then it costs more. The biggest issue with
this triangle-view of metrics is that “good” usually means yields, defects per million, or failure rates, which are all productivity rate metrics and not quality metrics. We need a more robust model to untangle quality metrics from productivity, timeliness, and financial metrics. We call this model the “Family of Measures.” We have used this model over the past three decades to help organizations evaluate existing metric systems; differentiate order winners from
market entry metrics; evaluate trends, articles, books, and claims; illustrate the Deming chain reaction and Taguchi loss function; explain why reliability metrics are quality metrics; create truly balanced scorecards; and even illustrate history. The sincere hope is that the Family of Measures reveals fresh insights into metric development and gives you an additional tool to make better data-based decisions. 

 

Four Distinct Metrics

Organizational performance metrics are like the periodic table in chemistry (Figure 1).

Figure 1 Periodic Table of Business Metrics

There are base elements, or metrics: productivity, financial, quality, and timeliness. These building blocks can be used as standalone metrics or combined to form compounds (i.e., rates and percentages). There are two easy ways to identify and separate the four metrics. Each metric  answers a different question and has a different unit-of measure. For example, if the unit-of-measure is currency (e.g., dollars, yen, euro), then the metric type is financial. If the unit-of-measure is an index (e.g., customer satisfaction, process capability index), then the metric type is quality. If the unit-of-measure is time (e.g., seconds, days, years), then
the metric type is timeliness. If the unit-of-measure is units, (e.g., counts, °F, grams), then the metric type is productivity. The second way to differentiate the four metrics is to understand what question is being asked. Financial metrics ask, “How much?” Productivity measures ask, “How many?” Timeliness metrics ask, “How fast?” Quality metrics ask, “How well?” All four base metrics can be either counts (attributes) or measures (variable). Table 1 is a summary of the two ways to differentiate the four metrics, including examples and definitions.

Of the four metrics, financial and timeliness metrics are the most intuitive. We have been counting money and telling time since grade school. It is the other two, productivity and quality, that give organizations trouble. Two interrelated problems exist. First, there is the common practice of defining quality with productivity, financial, or timeliness metrics. These substitutions and misconceptions hinder the creation of true quality metrics. Especially, they hinder the development of leading indicators of process quality that would improve variability, stability, and capability.

Table 1 Family of Measure Question Unit of Measure Examples
 

Family of Measure	Question	Unit of Measure	Examples
			Count Unlink			Measure
Productivity (units)	How many?	Units (calorie, meter, °C, liter, amp, cm3, psi, volt, kg, mole)	Scrap, rework, repair, pass, fail, throughput, waiting calls, returns, call abandonment, load, available rooms, fill level, capacity, available seat miles				Weight, height, length, temperature, volt, area, volume, pressure, distance, power, fuel economy, mass, energy, illumination, force, stress, current
Financial (currency)	How much?	$, €, ¥	Cost, profit, budget, assets, revenue, debt, income, sales, expenses	Taken to decimals (often treated as variable but remains an attribute for determining sample size)
Quality (Index)	How well?	Index	Survey scales (1, 2, 3, 4, 5) Customer satisfaction, employee satisfaction, patient satisfaction, student grades	Satisfaction survey slider scales (1-5) Control plan indices, patient safety index, blood pressure, process capability indices, Cp & Cpk
Timeliness (time units)	How fast?	sec, min, hour, day, week, month, year	Counts of whole months, weeks, days, or hours (1, 2, 3, …) Age, days late, seniority, tenure, block hours	Cycle time, handle time, duration, intervals, queue time, shipping time, response time, MTTF, MTBF, MTTR, lead time, delay, call time, setup time
Combinations (%)	At what rate?	rate, %	Labor rate, yield, return rate, load factor, return on assets, payout ratio, gross margin, mortality rate, occupancy %, average daily rate, revenue per room, advertising ROI, average daily rate, domestic book rate, website conversion rate, asset turnover ratio, stage length, profitability ratio, inventory turnover, speed

A Universal Concept

The four metrics are represented in many fields of study. Project management embeds the Family of Measures in its core knowledge areas: scope management, time management, cost management, and quality management (Project Management Institute, 2008). Scope, activity planning, and the work breakdown structure are productivity elements. Timeliness is represented by the schedule, displayed with Gant charts and networks. The project budget is the financial portion.  Variance indices represent quality efforts. Project risk management (FMEA) also contains the family of measures embedded in severity (quality), detection
(productivity), and occurrence (timeliness or productivity, depending on the scale). Logistics is another example of this universal concept. Logistics has five objectives: right product and place, right time or faster, right price/cost, and right condition (Stock and Lambert, 2001). The Family of Measures is a reoccurring schema that appears in all disciplines, although each may be worded differently. 

Productivity versus Quality Metrics 

The Taguchi Loss Function (Deming, 2000) is probably the best illustration of the difference between productivity and quality. A simple loss function is a step function between two states of a switch (on/off). The light is either on or off. Figure 2 illustrates production yields as a loss function.

Figure 2 Traditional View of Loss

Production, viewed in this way, is either on (100% good) or off (100% bad). We use the following questions to explore this widely-held view of production:

• Is there a “happy” zone?
• Is there a sense of loss all the time?
• Which Family of Measure is illustrated in the figure?
• What question does this metric answer?

There is absolutely a “happy zone” when things are deemed 100% good. No process improvement efforts will be taken under these “happy” conditions. There is no
feeling of loss under these conditions. When the product or service is deemed bad, a feeling of 100% loss exists, and problem-solving techniques are employed to regain control. This feeling of 100% good or 100% bad is binomial. It is a productivity metric. What we produced is either good or bad. How many good or bad? Yields are either calculated directly from counts; or measured, dispositioned, and then totaled as good or bad. Two problems occur as a result of this version of process management: (1) productivity yield metrics are used to monitor the process, masquerading as quality metrics; and, (2) problem solving is the predominant method used to “improve the process” when things are bad. 

An alternative to this process management strategy is illustrated in Figure 3.

Figure 3 Alternate View of Loss

Let us apply the same series of questions. There is no “happy zone” because the process aims to meet nominal. The odds of being exactly on nominal are small; thus, a sense of loss always exists and increases as we stray from nominal. This feeling leads to continuous process improvement efforts to center the process better and continuously reduce variability. Most importantly, improvement efforts continue even when the product or service is within tolerance (deemed
good). The primary question is, “How good or bad?” This question is a true quality metric. What we produce may be either good or bad, but we also want to know the degree of goodness or badness or to answer the question, “How well?” To do this, we measure variability, central tendency, and the stability of a process. We compare our process to customer specifications and calculate process capability and process management indices. The idea is not to reach an arbitrary goal, but to continuously improve the process.

Two profound differences occur as a result of this version of process management: (1) quality indices are used to monitor the health of the process (with throughput, yields, cycle time, costs, and other metrics), and, (2) continuous improvement is the method used to improve the process. To appreciate the difference between the two competing views of process management, one would need to understand all the organizational factors that promote the traditional productivity-focused step function version (the “hero” mentality, competitive reward systems, discomfort with statistics, overreliance on audits and inspections); a discussion of which is well beyond the scope of this article. 

Need for Order Winners 

Most companies operate on a half-set of metrics (productivity and financial), sprinkling in some operational-level timeliness indicators, but mostly excluding quality altogether. They think they are measuring quality, mostly with yields, and then wonder why customer complaints and warranty costs are rising, while market share and repeat business are declining. The ever-increasing presence of global competition is not going away. Our needs are urgent: the need to stop degrading rich timeliness and quality metrics into simple percentages; the need to differentiate between productivity and quality; and the need to develop leading
indicators of quality. Family of Measures is a tool that helps address these needs through the reexamination and delineation of metrics into distinct metric families.

References

Project Management Institute. A Guide to the Project Management Body of Knowledge (PMBOK Guide), fourth edition. Newtown Square, Pennsylvania; Project Management Institute, Inc., 2008.
Stock, J., and Lambert, D. Strategic Logistics Management, fourth edition. New York: McGraw-Hill Irwin, 2001.
Deming, W. Edwards. The New Economics for Industry, Government, Education, second edition. Cambridge: MIT Press. 2000.

Dr. David Wetzel

Dave Wetzel is a Lean Six Sigma Master Black Belt with over 30 years of work experience in the industrial, service, and public sectors as a process improvement
engineer, facilitator, trainer, and course developer. He has a Ph.D. in Industrial and Systems Engineering from The Ohio State University, an M.S. in Engineering Management from Rensselaer Polytechnic University, and a B.S. in Chemical Engineering, also from OSU. Dave has held several different ASQ certifications:
quality engineer, black belt, and quality technician. Dave has worked in Europe, Asia, the Middle East, and the Americas with such organizations as Dell, TVA, IBM, GE, Chemetall, RIT, Nationwide Insurance, Samsung, Eli Lilly, DeCA, the George Group, McKee Foods, the U.S. Army, and Xerox. Dave is responsible for the Manufacturing Systems Engineering program at Weber State University. Dave can be reached at dwetzel73@gmail.com or 585.752.2758.