# Quality Software & Analysis: Is Your Process Performing?

June 27, 2011

Discover the most common SPC errors and learn how to correct them.

One of the real treats of working in a statistical process control (SPC) software company is the exposure to a wide range of SPC applications. The use of process performance and capability indices is a consistently top-ranking concern for our client-base, which extends globally across tens of thousands of clients in all types of industries.

While I’ve enjoyed discussing these issues with so many clients over the years, in detailed training sessions or simply in response to their technical support queries, I feel at times the bearer of some rather unfortunate news:

If your suppliers currently provide these estimates, demand either 100% inspection (for example, a measured value for each and every item shipped), or evidence of process control and its corresponding capability estimate.

Note the only mathematical difference between the two process capability indices and their corresponding performance indices is the use of process sigma in place of sample sigma. Sample sigma is the standard deviation of a sample such as calculated by a handheld calculator or Excel’s STDEV function; process sigma is the standard deviation of the process, calculated using a control chart.

Error: Using sample sigma for process capability index.

Proper approach: Use process sigma for process capability estimate.

Figure 1 contains an example control chart, the classic Shewhart X-bar chart (at top) with its accompanying range chart (at bottom). In this example, the X-bar chart is estimating process location using the average of a five-piece subgroup at each time period; the range chart estimates process variation using the difference between the maximum and minimum measurements in the five piece sample. The choice of subgroup size is based on a number of considerations, including economics, desired sensitivity in detecting process shifts and the dynamics of the process itself, which dictates a rational subgroup.

A proper control chart plots the data in time sequence-the data on the far left-hand side of the chart is from the earliest time period, and each subsequent value to the right corresponds to a later time period. In this way, a control chart is perfectly suited for processes, which, by their very nature, occur over time. Each subgroup represents a process sample at a given point in time, and provides an estimate of the “short-term” process location and variation.

The control limits are calculated using the average of all the short-term (for example, subgroup) estimates. Process control is thus synonymous with process stability-stability is evident when the short-term estimates provide a reliable prediction of longer-term behavior. This explains why it is critical for the data to be plotted in the order they occurred in the process; otherwise, the short-term estimates are flawed.

The control chart is further enhanced using run test rules to look for trends within the control limits, providing additional evidence of process instability.

Error: Neglecting the order of the data, or plotting random samples from a batch where the sequence of production is unknown.

Proper approach: Plot the data on the control chart in the order it occurred in the process. This is easier to do if the data is recorded and analyzed in real-time by the process personnel.

The process sigma value in the capability equations is based on the average range (R-bar), so that process capability relies on accurate estimates of both the grand average x-doublebar and the average range R-bar . If the range chart is out of control, then a single estimate for R-bar is not meaningful-the process variation is not stable; it is changing and cannot be estimated by a single value. In this case, meaningful control limits cannot be established for the X-bar chart.

If the range chart is in control, then R-bar is a reliable estimate of the process variation, and is used for calculating the X-bar control limits. If the X-bar chart is out of control, then the process location is unstable and similarly cannot be estimated by a single average value.

When both charts are in control, the natural variation inherent to the process may be estimated. This is the level of variation expected from the process over time. This estimate of common cause variation is improved with additional data. Notice how any single subgroup, or interval of 5 or 6 subgroups, do not provide a full picture of the common cause variation in Figure 1. Sufficient data-at least 25-30 subgroups of size five; more subgroups for smaller subgroups-are needed over a sufficient time period to experience the common causes inherent to the process.

Error: Reporting capability index for a short time interval, such as a single shipment.

Proper approach: Update the capability index only when the process undergoes a sustained shift. A process in control has, by definition, a single capability index.

It is uncanny how often practitioners choose to present their capability indices with histograms instead of control charts. Note the histogram shown in Figure 2, obviously from a well-controlled process. Or is it?

The data used to generate Figure 2’s histogram is shown on the control chart of Figure 3. The time sequence of the data is ignored in the histogram, thus preventing proper analysis of stability. For a set of sample data, the histogram is both misleading and meaningless, without an accompanying control chart.

Error: Reporting capability index when the process is not in control, or its stability is unknown.

Proper approach: Always report the process capability index with the accompanying control chart that verifies the state of process control. A capability index should be considered unreliable without the control chart used to estimate its parameters.

A process that is stable may be approximated by a single distribution. A process that is out of control has multiple distributions. A special cause, referred to as a process shift when it is sustained, is evidence of a new process distribution. The special cause has the effect of changing one or more parameters of the in-control distribution: its location (estimated by average or median), spread (estimated by standard deviation) and shape (including skewness and kurtosis).

While some statisticians suggest the normal distribution is prevalent, real-world processes are designed to meet specific needs and may be sufficiently non-normal by design-mechanical stops are introduced to truncate the output; process improvement efforts move the process center closer to an optimal value; measurement scales (such as pH) may be inherently non-normal; measurements may be bounded by a physical limitation (flatness must be a positive value).

Once statistical control is established, a meaningful statistical distribution may be fit using SPC software.

The Kolmogorov-Smirnov (K-S) goodness of fit test is typically used to pick the best fit distribution from the possible distributions. If the distribution is sufficiently non-normal, the standard process capability calculations, which are only applicable to a normal distribution, must be adjusted. For example, the 0.00135 and 99.865 percentiles of the fitted distribution may be used to determine the comparable non-normal Cp.

Error: Applying the standard process capability calculations to process data that is not well-approximated by a normal distribution.

Proper approach: Once statistical control is established, use the K-S statistic to determine the best distributional fit. Adjust the standard equations to provide equivalent levels of protection for non-normal distributions.

“New Approaches to SPC”

Podcast: “An Overview of SPC and Justifying a Software Expense”

“Six Steps to Shop Floor Acceptance of SPC Software”

Most process performance estimates (and some process capability estimates) are essentially useless to your customers.

A process and its statistics are only predictable if the process is stable; an unstable process is chaotic and unpredictable.

Once statistical control is established, a meaningful statistical distribution may be fit using SPC software.

One of the real treats of working in a statistical process control (SPC) software company is the exposure to a wide range of SPC applications. The use of process performance and capability indices is a consistently top-ranking concern for our client-base, which extends globally across tens of thousands of clients in all types of industries.

While I’ve enjoyed discussing these issues with so many clients over the years, in detailed training sessions or simply in response to their technical support queries, I feel at times the bearer of some rather unfortunate news:

*Most process performance estimates (and some process capability estimates) are essentially useless to your customers. They provide no indication of your process’s ability to meet requirements over the long term; they provide no comfort regarding a given shipment’s conformance to requirement.*If your suppliers currently provide these estimates, demand either 100% inspection (for example, a measured value for each and every item shipped), or evidence of process control and its corresponding capability estimate.

To understand the reasoning behind the use of process performance, a quick review of the indices is warranted. The calculations typically used are displayed in the equation above, where USL is the upper specification limit (for example, the largest acceptable value for any unit); LSL is the lower specification limit (for example, the smallest acceptable value for any unit); the symbol x with the two lines over top is the process average, sometimes referred to as x double-bar; σ

_{sample}is the sample standard deviation; and σ_{process}is the process standard deviation.Note the only mathematical difference between the two process capability indices and their corresponding performance indices is the use of process sigma in place of sample sigma. Sample sigma is the standard deviation of a sample such as calculated by a handheld calculator or Excel’s STDEV function; process sigma is the standard deviation of the process, calculated using a control chart.

Error: Using sample sigma for process capability index.

Proper approach: Use process sigma for process capability estimate.

Figure 1 contains an example control chart, the classic Shewhart X-bar chart (at top) with its accompanying range chart (at bottom). In this example, the X-bar chart is estimating process location using the average of a five-piece subgroup at each time period; the range chart estimates process variation using the difference between the maximum and minimum measurements in the five piece sample. The choice of subgroup size is based on a number of considerations, including economics, desired sensitivity in detecting process shifts and the dynamics of the process itself, which dictates a rational subgroup.

A proper control chart plots the data in time sequence-the data on the far left-hand side of the chart is from the earliest time period, and each subsequent value to the right corresponds to a later time period. In this way, a control chart is perfectly suited for processes, which, by their very nature, occur over time. Each subgroup represents a process sample at a given point in time, and provides an estimate of the “short-term” process location and variation.

The control limits are calculated using the average of all the short-term (for example, subgroup) estimates. Process control is thus synonymous with process stability-stability is evident when the short-term estimates provide a reliable prediction of longer-term behavior. This explains why it is critical for the data to be plotted in the order they occurred in the process; otherwise, the short-term estimates are flawed.

The control chart is further enhanced using run test rules to look for trends within the control limits, providing additional evidence of process instability.

Error: Neglecting the order of the data, or plotting random samples from a batch where the sequence of production is unknown.

Proper approach: Plot the data on the control chart in the order it occurred in the process. This is easier to do if the data is recorded and analyzed in real-time by the process personnel.

The process sigma value in the capability equations is based on the average range (R-bar), so that process capability relies on accurate estimates of both the grand average x-doublebar and the average range R-bar . If the range chart is out of control, then a single estimate for R-bar is not meaningful-the process variation is not stable; it is changing and cannot be estimated by a single value. In this case, meaningful control limits cannot be established for the X-bar chart.

If the range chart is in control, then R-bar is a reliable estimate of the process variation, and is used for calculating the X-bar control limits. If the X-bar chart is out of control, then the process location is unstable and similarly cannot be estimated by a single average value.

When both charts are in control, the natural variation inherent to the process may be estimated. This is the level of variation expected from the process over time. This estimate of common cause variation is improved with additional data. Notice how any single subgroup, or interval of 5 or 6 subgroups, do not provide a full picture of the common cause variation in Figure 1. Sufficient data-at least 25-30 subgroups of size five; more subgroups for smaller subgroups-are needed over a sufficient time period to experience the common causes inherent to the process.

Error: Reporting capability index for a short time interval, such as a single shipment.

Proper approach: Update the capability index only when the process undergoes a sustained shift. A process in control has, by definition, a single capability index.

It should be obvious that a process and its statistics are only predictable if the process is stable; an unstable process-such as one influenced by special causes-is chaotic and unpredictable. If the process is out of control, the average, average range, and resulting process sigma and capability statistics include the effects of special causes, which add a statistical bias to their estimate.

It is uncanny how often practitioners choose to present their capability indices with histograms instead of control charts. Note the histogram shown in Figure 2, obviously from a well-controlled process. Or is it?

The data used to generate Figure 2’s histogram is shown on the control chart of Figure 3. The time sequence of the data is ignored in the histogram, thus preventing proper analysis of stability. For a set of sample data, the histogram is both misleading and meaningless, without an accompanying control chart.

Error: Reporting capability index when the process is not in control, or its stability is unknown.

Proper approach: Always report the process capability index with the accompanying control chart that verifies the state of process control. A capability index should be considered unreliable without the control chart used to estimate its parameters.

A process that is stable may be approximated by a single distribution. A process that is out of control has multiple distributions. A special cause, referred to as a process shift when it is sustained, is evidence of a new process distribution. The special cause has the effect of changing one or more parameters of the in-control distribution: its location (estimated by average or median), spread (estimated by standard deviation) and shape (including skewness and kurtosis).

While some statisticians suggest the normal distribution is prevalent, real-world processes are designed to meet specific needs and may be sufficiently non-normal by design-mechanical stops are introduced to truncate the output; process improvement efforts move the process center closer to an optimal value; measurement scales (such as pH) may be inherently non-normal; measurements may be bounded by a physical limitation (flatness must be a positive value).

Once statistical control is established, a meaningful statistical distribution may be fit using SPC software.

The Kolmogorov-Smirnov (K-S) goodness of fit test is typically used to pick the best fit distribution from the possible distributions. If the distribution is sufficiently non-normal, the standard process capability calculations, which are only applicable to a normal distribution, must be adjusted. For example, the 0.00135 and 99.865 percentiles of the fitted distribution may be used to determine the comparable non-normal Cp.

Error: Applying the standard process capability calculations to process data that is not well-approximated by a normal distribution.

Proper approach: Once statistical control is established, use the K-S statistic to determine the best distributional fit. Adjust the standard equations to provide equivalent levels of protection for non-normal distributions.

When a process is out of control, or its stability is unknown, a typical recommendation is the use of the process performance indices. This method suffers from a fundamental flaw-a single estimate of a parameter is unreliable for an unstable process. When the process is out of control, the sample standard deviation will include the effects of one or more distributions, the impact of which is dependent on the relative number of samples from each distribution, as well as the quantitative differences between the distributions. Each distribution may or may not be well approximated by a normal distribution, making the standard calculations irrelevant. Deming discussed his All or Nothing Rule for determining the economic sample size based on the cost of undetected errors. For unstable processes, he invoked the Orsini modification, which essentially led to 100% inspection for critical characteristics. Error: Assuming a random sample will adequately quantify an unpredictable process. Proper approach: 100% inspection of all units in the shipment, or proper process capability analysis as described above.

**Q**## QUALITY ONLINE

For more information on statistical process control, visit www.qualitymag.com for the following:“New Approaches to SPC”

Podcast: “An Overview of SPC and Justifying a Software Expense”

“Six Steps to Shop Floor Acceptance of SPC Software”