Quality Magazine logo
search
cart
facebook twitter linkedin youtube
  • Sign In
  • Create Account
  • Sign Out
  • My Account
Quality Magazine logo
  • NEWS
  • PRODUCTS
    • FEATURED PRODUCTS
    • SUBMIT YOUR PRODUCT
  • CHANNELS
    • AUTOMATION
    • MANAGEMENT
    • MEASUREMENT
    • NDT
    • QUALITY 101
    • SOFTWARE
    • TEST & INSPECTION
    • VISION & SENSORS
  • MARKETS
    • AEROSPACE
    • AUTOMOTIVE
    • ENERGY
    • GREEN MANUFACTURING
    • MEDICAL
  • MEDIA
    • A WORD ON QUALITY PUZZLE
    • EBOOK
    • PODCASTS
    • VIDEOS
    • WEBINARS
  • EVENTS
    • EVENT CALENDAR
    • IMTS
  • DIRECTORIES
    • BUYERS GUIDE >
      • Supplier Insights
    • NDT SOURCEBOOK
    • VISION & SENSORS
    • TAKE A TOUR
  • INFOCENTERS
    • Digital Quality Management Systems
    • NEXT GENERATION SPC & QUALITY ANALYTICS
  • AWARDS
    • ROOKIE OF THE YEAR
    • PLANT OF THE YEAR
    • PROFESSIONAL OF THE YEAR
  • MORE
    • Expert Columns
    • NEWSLETTERS
    • QUALITY STORE
    • INDUSTRY LINKS
    • SPONSOR INSIGHTS
  • EMAG
    • eMAGAZINE
    • ARCHIVES
    • CONTACT
    • ADVERTISE
  • SIGN UP!
SoftwareManagement

Software & Analysis

Got 10,000 Data Points?

Rethinking the SPC workflow for large-scale data.

By Steve Daum
April 15, 2013
man working in high tech manufacturingJust as a recipe designed for four may require some changes if it is called upon to serve 4,000, dramatically up-scaling the number of quality metrics being monitored may demand new approaches when deploying SPC.
 
To be sure, the techniques of statistical process control (SPC) have been in use for nearly a century. In the early 1920s, when Walter Shewhart pioneered these tools, they were deployed and maintained with pencil and paper. The benefits of early detection of variation were apparent despite the manually intensive work required. The recipe, so to speak, was straightforward and easy to follow.
 
Today, computers and software are widely used for SPC, but the basic workflow of SPC has remained the same: 1) measure something, 2) visualize it on a chart, 3) learn from and possibly react to signals that are detected. One problem with this workflow is the difficulty of scaling to the large number of metrics required in today’s competitive and sometimes highly regulated environments.
 
In the January 26-27, 2013, weekend edition of The Wall Street Journal, Bill Gates writes about the importance of data collection, harkening back to the innovators of the 19th century, where “measuring tools…allowed inventors to see if their incremental design changes led to improvements.” Gates himself remarks about how he has been struck by “how important measurement is to improving the human condition.” Citing data such as that related to the reduction of polio cases from 350,000 in 1988 to 212 worldwide in 2012, Gates points out how critical measurement is to understanding progress. 
 
Those in manufacturing, as well as healthcare and service industries, know that this is true. Measurement data is fundamental to every process. While there can be no doubt of this, using the data in a way that it yields information depends on being able to respond to it. One can, as the saying goes, be “drowning in data but starved for information.”
 
Imagine any moderately complex manufacturing production line. Think about the number of metrics that might be gathered along the line. Even in focusing only on metrics that have a direct impact on quality, the numbers can be staggering. What if there are 300, 500 or even 1,000 metrics that need to be monitored for variation? Is it realistic or practical to deploy 1,000 control charts along this hypothetical line? In most cases, even when computers and software are available, the answer will be no. It is just not practical. Who would look at these charts? How many workers could be spared to do this SPC work? 
 
There are at least two barriers to scaling SPC to these levels: 1) the effort/cost required to set up the charts initially; and 2) the attention burden on the workers who must look at, think about and react to these charts. To truly scale SPC charting efforts, both of these must be addressed.
 
The first issue clearly represents a challenge to any large organization. Setting up hundreds of charts will demand time and effort. The employees doing this work require clear training related to SPC charting, understanding variation, and analyzing charts. Knowledge of product specifications and process parameters is also important. Errors made when setting up these charts will ultimately affect the payback time for the investment in SPC charting.
 
The second issue, the problem of paying attention to the charts, is also difficult to address. In the production settings with which we are familiar, workers are busy. They tend not to have free time, time when they can focus their attention on a chart, engage in analytical thinking, react to it correctly, and then get back to what they were doing.
 
So, we agree that scaling SPC is difficult. Does that mean we should not try to do it? The answer, unfortunately, is “it depends.” There are many production situations where a large scale SPC charting effort may not yield a payback. However, at today’s increasingly faster rates of production, there are other cases where large scale SPC charting can create a return on investment. 
 
Think of any production line capable of producing 5,000 pieces in one hour. This is not uncommon. On customer visits, we’ve seen these and others with even higher rates of production. Depending on the price of the unit being made, if an hour goes by where unwanted variation in a key metric goes undetected, the cost of this scrap can be substantial. Hourly sampling plans are common; how much faulty product can be produced in one hour? High speed production is great, but even when you reduce sampling time to, say, every 20 minutes, the risk of producing large amounts of scrap cannot be overlooked.
 
Here are some thoughts and approaches to overcoming the barriers to scaling your SPC charting efforts.
 
First of all, whenever possible, your data collection efforts should be automated. When your production line is running, product should be being measured and tested and these results should be flowing into databases. With today’s modern production machines, this is often possible without operator intervention. When manual measurement or testing is required, the sampling rates should be set based on the potential cost of failures. Additionally, the manual measurement and testing workflow should be well integrated into some worker’s natural workflow. These manually gathered results should flow directly into a database rather than onto paper and later put into the database.
 
Once data from your process is flowing into a database, software-based control charts of the key metrics should be defined. The employees doing this work need to be familiar with SPC concepts, including control limits, out-of-control testing, and chart type selection. Ideally, the charts should be defined so that each time a viewer sees it, it will be showing a reasonable amount of the most recent, most relevant data—and no further chart configuration is needed. The goal for these charts should be “define them once, view them often.”
 
Defining these charts can be time consuming. Think about this workflow and ways to make it more efficient. For example, have “template charts” pre-defined. When a new chart is needed, rather than starting from scratch, use a template, give it a new name and change only the settings that are different. Creating and using meaningful naming conventions is important. Since the number of charts can be large, good naming conventions can reduce the cognitive burden on employees who commonly look for and view the charts.
 
Next, what about looking at all these charts? Here is where new technology and new thinking about the SPC charting workflow might really improve the scalability of SPC. 
 
What is the reason for looking at a control chart? To see a signal. By the time a control chart is set up, you really should not expect to see too many signals. The limits were established when the process was running in a predictable, in-control state. 
 
What we propose is that operators see control charts only when a signal is detected, preferably by software. There might be several hundred control charts defined. However, the appropriate operator is shown a control chart only when a signal is detected by some other agent. The agent in this case is the SPC software. While operators go about their work, the software is patiently watching data flow into the database, applying SPC thinking to this data, then raising a signal for operator intervention only when needed. 
 
There are very real challenges when scaling SPC charting efforts. However, if we are willing to change our paradigm and think carefully about the workflows, these challenges can be overcome and large scale SPC charting efforts can pay for themselves by reducing scrap and variation. 
 
While measurement tools used to track polio vaccinations might be different from those used in a manufacturing facility, there can be no doubt that tools for understanding data are essential to making sense of, and responding to, critical metrics. Bill Gates asserts that the measurement systems used in Africa to track vaccination efforts “will help us to finish the job of polio eradication within the next six years.” Surely, the measurement tools available to manufacturing facilities to understand data flowing into a database can also have dramatic impacts on outcomes and process improvement.
KEYWORDS: PQ Systems statistical process control (SPC)

Share This Story

Looking for a reprint of this article?
From high-res PDFs to custom plaques, order your copy today!

Recommended Content

JOIN TODAY
to unlock your recommendations.

Already have an account? Sign In

  • 2024 Quality Rookie of the Year Justin Wise 1440x750px banner with "Quality Rookie of the Year" logo inset

    Meet the 2024 Quality Rookie of the Year: Justin Wise

    Justin Wise is an exceptional individual who has been...
    Aerospace
    By: Michelle Bangert
  • Man with umbrella and coat stands outside while it rains at night looking at a building.

    Nondestructive Testing: Is there an ethics problem?

    I was a whistleblower who exposed fraudulent activities...
    NDT
    By: Dale Norwood
  • Unraveling Deflategate: Football stadium with closeup of football on field

    Unraveling the Tom Brady Deflategate

    The Deflategate scandal erupted following the 2014 AFC...
    Measurement
    By: Greg Cenker and Henry Zumbrun
Manage My Account
  • eMagazine Subscriptions
  • Newsletters
  • Online Registration
  • Subscription Customer Service
  • Manage My Preferences

More Videos

Sponsored Content

Sponsored Content is a special paid section where industry companies provide high quality, objective, non-commercial content around topics of interest to the Quality audience. All Sponsored Content is supplied by the advertising company and any opinions expressed in this article are those of the author and not necessarily reflect the views of Quality or its parent company, BNP Media. Interested in participating in our Sponsored Content section? Contact your local rep!

close
  • This image shows a person seated next to a Bobcat T66 compact track loader.
    Sponsored byPolyWorks by InnovMetric

    Supercharging Digital Gauging at Bobcat North America

  • Dorsey Calibration Lab photo by Tom LaBarbera Picture this Studios
    Sponsored byDorsey Metrology International

    Ensuring Product Quality in a Competitive Manufacturing Landscape

  • This image displays a Eddyfi Technologies Cypher portable inspection instrument alongside a scanner for non-destructive testing (NDT).
    Sponsored byEddyfi Technologies

    A Safer, Smarter Approach to Weld Inspection: Why Advanced Ultrasonic Testing Is Redefining Industry Standards

Popular Stories

MicroRidge MobileCollect wireless measurement system

Before AI Can Help, the Data Has to Be Ready

a titanium diaphragm speaker driver

The One Thing Elon Gets Right Is Designed to Scare You

This image shows a person seated next to a Bobcat T66 compact track loader.

Supercharging Digital Gauging at Bobcat North America

2026 Quality Professional of the Year!

Events

June 9, 2026

Future-Proof your Quality Processes with Advanced 3D Optical CMM Technology

Discover how to effortlessly capture complex data, leverage true multi-sensor automation, and ensure continuous operation without creating inspection delays.

June 22, 2026

Automate 2026

Automate is North America's largest robotics and automation event — and the best place to take your ideas from insight to impact.
 
Our show floor features the world’s leading automation solutions, from AI and robotics to motion control, vision systems, and more. Plus, our educational conference is second to none, led by the brightest minds in automation today.
 
Ready to transform the way you work? Take the next step at Automate.
View All Submit An Event

Products

Lean Manufacturing and Service Fundamentals, Applications, and Case Studies

Lean Manufacturing and Service Fundamentals, Applications, and Case Studies

See More Products
Quality Podcast Channel Custom Content

Related Articles

  • Automate

    Entries Open for $10,000 Cash Prize at Automate 2022 in ‘The Cowen Startup Challenge: Automate to Outperform’

    See More
  • Got a Problem? Try Our Reader Forum for Help

    See More
  • The DVM6 digital microscope

    What You Always Wanted to Know About Digital Microscopy, but Never Got Around to Asking

    See More

Related Products

See More Products
  • H1585-Mawby_cover_border.jpg

    NAVIGATING BIG DATA ANALYTICS

  • Reverse Engineering: 3D Data Capture DVD

See More Products

Related Directories

  • Mark-10 Corp.

    Mark-10 engineers and manufactures force and torque measuring gauges, systems, and accessories. From peel testing to tensile testing, from spring testing to weld strength testing, we help Quality Control and R&D professionals around the globe assess and ensure product quality. Our products are made in the USA and are covered by a 3-year warranty with free lifetime support.
×

Stay in the know with Quality’s comprehensive coverage of
the manufacturing and metrology industries.

Newsletters | Website | eMagazine

JOIN TODAY!
  • RESOURCES
    • Advertise
    • Contact Us
    • Directories
    • Manufacturing Division
    • Store
    • Want More
  • SIGN UP TODAY
    • Create Account
    • eMagazine
    • Newsletters
    • Customer Service
    • Manage Preferences
  • SERVICES
    • Marketing Services
    • Market Research
    • Reprints
    • List Rental
    • Survey/Respondent Access
  • STAY CONNECTED
    • LinkedIn
    • Facebook
    • YouTube
    • X (Twitter)
  • PRIVACY
    • PRIVACY POLICY
    • TERMS & CONDITIONS
    • DO NOT SELL MY PERSONAL INFORMATION
    • PRIVACY REQUEST
    • ACCESSIBILITY

Copyright ©2026. All Rights Reserved BNP Media, Inc. and BNP Media II, LLC.

Design, CMS, Hosting & Web Development :: ePublishing