Data Collection War Stories
The most unpredictable factor in almost every process? People.
Over the course of 20 years, I’ve visited hundreds of different factory floors and seen a wide variety of manufacturers implement data collection and statistical process control (SPC) solutions. I’m often called upon to help establish the proper monitoring strategy to detect changes in a process while allowing for normal variation. Experience has led me to pay particularly close attention to the impact of the most unpredictable factor in almost every process: human beings. Humans are a ripe source of special cause variation, system breakdown or just humorous misunderstandings. I have seen how, sometimes innocently and sometimes deliberately, human actions can result in unwanted outcomes, missed opportunities or head-scratching mess-ups. I thought I would share some of the more interesting or amusing situations that I’ve seen over the years.
The Too Consistent Situation
The quality manager at an injection molding company had put into place a sampling plan that ensured coverage of all cavities for each production run. He was very clear to line operators that he expected the plan to be faithfully followed. But to get the quality manager off his back, one of the operators kept a stash of “good” products for each run to measure at the appointed hour.
Software can help avoid some data collection issues.
Implementing an automated inspection process eliminates the need for operators to input manual readings.
This allows for accurate first inspection records and a re-inspection program that works as intended.
Resolution: We analyzed the data and found a distinct lack of variation during this operator’s shift. When he discovered that his stash of trustworthy products had mysteriously disappeared, the operator’s adherence to the sampling plan was flawless.
The Inconsistency Situation
One quality conscious automotive supplier upgraded their old torque guns with new, automated torque guns that were intended to improve bolt tightening precision throughout assembly. But the results were unexpectedly inconsistent. Measurements from the new torque guns were all over the place and some of the bolts were tightened too much. At the time, the data from each torque gun was stored locally.
Resolution: We collected and combined the data from each individual torque gun and performed comparative analysis of all the data by bolt and by operator. The problem stood out like a beacon on the charts: where we expected to see one torque value per bolt, we saw two or three. And we could ID the change in a shift by the change in the control charts. The problem was with certain operators who were still “pumping the torque guns just to make sure the bolts were tight”—a common practice with the older model guns but a counterproductive practice with the new ones. With this “opportunity for retraining” identified, much more precise torque control for the bolts was quickly achieved.
The Avoidance Situation
A company making carbon fiber components wanted to promote careful re-inspection and analysis when any failure was detected. They developed an elaborate re-inspection procedure with several layers of review and sign off. Initially, they noticed a decrease in the number of reported failures and were pleased with the outcomes of this initiative. But they soon discovered that their product quality had not improved, and in fact had gotten worse. We conducted a review and discovered an unfortunate side effect of the re-inspection program: in order to avoid the onerous re-inspection procedures, operators admitted that they were giving “any close calls” the “benefit of the doubt” and letting questionable product pass first inspection.
Resolution: We implemented an automated inspection process and eliminated the ability of operators to input manual readings. From then on, first inspection was accurately recorded and the re-inspection program operated as envisioned.
The Looks Pretty Situation
The quality manager at a metal fabricator was quite proud of the Certificate of Analysis (CofA) he had configured his software package to automatically produce at the end of a run. He showed me how his company’s logo was paired with the logo of the customer; the charts were smartly color-coded and the statistics neatly set out in rows. When I pointed out that the CofA he handed me actually revealed that the stamping process in question was not very capable, he said that it really didn’t matter: “[The customer] just checks to see that the CofA is there with the shipment; if the document looks good they don’t even check it.”
Resolution: It turned out that the customer in question DID check the CofAs. We ended up modifying the report with additional colors to alert the quality manager of any problems in a lot before a shipment was made to a customer. The quality manager also successfully used the analysis tools of the software package to uncover and investigate production issues.
The Over-Does-It Situation
A consumer products manufacturer was trying to apply SPC analysis to weight data collected from a bottle filling line. In reviewing their inspection practices, we discovered that the filling station inspectors were correcting perceived problems by hand-filling underweights and dumping out overfills before recording the readings.
Resolution: We conducted an SPC Kaizen and demonstrated how reacting to normal variation amplifies noise in the manufacturing process and impairs the usefulness of the data collected. The team agreed to run the process without the constant manual correction. Absent the human intervention, we were able to discover that select filler nozzles were out of specification. When that was corrected, the data further revealed large variations between batches. Ultimately, it was necessary to move the SPC efforts upstream to the batch mixing process—and the filling station inspectors reduced their workload!
The Small Situation
At a juice manufacturer, an inspection station operator was responsible for selecting several random samples for weight control testing at designated intervals. The problem was that “random” sampling is never truly “random” when selection is left to human discretion. The operator at issue stood less than 5 feet tall and had a hard time sampling from the far side of the line. Without any ill intent, sampling tended to be taken from nearby product.
Resolution: We developed a structured sampling protocol that eliminated operator discretion in the sampling process—and provided a tool for the operator to gather samples from all areas of the line. Only after putting this new procedure and assistance in place did the company discover that fill heads in one of the quadrants of a filling machine were problematic.
The Million Dollar Situation
An executive at a powder aerosol manufacturer explained that his production team didn’t need to meet with me because they felt that their processes were already as finely tuned as reasonably possible. He was confident that there was little room for improvement with his current operation and didn’t feel the need to implement SPC.
Resolution: Because his team had achieved the “holy grail” of quality, I asked if he’d be willing to share some of his data with me so I might have a better understanding of how they did it. He provided material and manufacturing cost information along with their production data. I was indeed impressed with how little variation there was in the processes I reviewed, but I noticed that they were still “giving away” a significant percentage of their product by over-filling aerosol cans. I conducted and sent them the results of a cost analysis which demonstrated that they could shift their process target and, given their large sales volume, save close to $1 million a year without worrying about falling below the stated weight. They took the meeting.
The Bonus Situation
A paint manufacturer received a call from an irate customer complaining that upon opening a rush shipment of paint, they found the cans were all filled with water. It turned out that the production manager was eligible for a bonus if he got the shipment out to the customer on time, but he knew that the batch being readied for shipment didn’t meet the customer’s specifications and would be rejected. He pressured the quality manager to release the shipment anyway. The quality manager, not feeling morally right in releasing the shipment as is, but also not having the authority to overrule the production manager, decided to swap the bad product with water. His thinking was that the customer would simply dump the water down the drain (avoiding any return costs), and the quality manager would ready a second shipment containing good product. Unfortunately for them, the customer called and complained directly to the owner and the jig was up!
Resolution: The production manager and quality manager were fired. The company hired new managers and changed their procedures so that the quality department had the final say in shipments to customers. Working with the new team, we significantly improved the company’s manufacturing processes.