Quality Magazine logo
search
cart
facebook twitter linkedin youtube
  • Sign In
  • Create Account
  • Sign Out
  • My Account
Quality Magazine logo
  • NEWS
  • PRODUCTS
    • FEATURED PRODUCTS
    • SUBMIT YOUR PRODUCT
  • CHANNELS
    • AUTOMATION
    • MANAGEMENT
    • MEASUREMENT
    • NDT
    • QUALITY 101
    • SOFTWARE
    • TEST & INSPECTION
    • VISION & SENSORS
  • MARKETS
    • AEROSPACE
    • AUTOMOTIVE
    • ENERGY
    • GREEN MANUFACTURING
    • MEDICAL
  • MEDIA
    • A WORD ON QUALITY PUZZLE
    • EBOOK
    • PODCASTS
    • VIDEOS
    • WEBINARS
  • EVENTS
    • EVENT CALENDAR
    • IMTS
  • DIRECTORIES
    • BUYERS GUIDE >
      • Supplier Insights
    • NDT SOURCEBOOK
    • VISION & SENSORS
    • TAKE A TOUR
  • INFOCENTERS
    • Digital Quality Management Systems
    • NEXT GENERATION SPC & QUALITY ANALYTICS
  • AWARDS
    • ROOKIE OF THE YEAR
    • PLANT OF THE YEAR
    • PROFESSIONAL OF THE YEAR
  • MORE
    • Expert Columns
    • NEWSLETTERS
    • QUALITY STORE
    • INDUSTRY LINKS
    • SPONSOR INSIGHTS
  • EMAG
    • eMAGAZINE
    • ARCHIVES
    • CONTACT
    • ADVERTISE
  • SIGN UP!
MeasurementTest & Inspection

Prisoners of Precision

Precision costs money.

By Hill Cox
July 15, 2019

Readers of this column are a gentle lot and not likely to run afoul of criminal law but can end up in a similarly frustrating situation that limits their working life. Instead of steel bars restraining them, specifications on setting masters will lock them up and they will only have themselves to blame. The purpose of this column is prevention so you don’t become a prisoner of precision.

The mistake too many folks make is specifying tolerances for dimensional masters that are not realistic thus causing escalating despair and usually increased costs.

 Theoretically, the more precise a master is, the better it is, but along with that, the more expensive it will be. Sometimes a tolerance is specified for a master without looking at the direction the tolerance specified will take. For example, if you want a master that is within 20µ” or .0005mm of the specified size, a chart in a gage maker’s catalog will tell you it is XX tolerance. But those charts represent unilateral tolerances ie; all one way plus or minus from the size you want. If the gage maker knows the tolerance is for a master, they will split it in half and apply it bilaterally so if you had specified an X tolerance of 40µ” or .001mm, you would get what you want at a lower cost.

There are more cost increases to all of this beyond the purchase price of the master. Overly precise masters will wear out of limit faster than need be so you’ll be replacing them more often. Such masters are a waste of money if they will be used in an unstable temperature environment or the parts being measured are literally hot off the machine that produced them.

Precision costs money. The more precise you want a master to be, the more it will cost you to get it, but there is a way to reduce the cost: order a lower tolerance but have it calibrated and use the actual calibrated size for setting purposes. This may require any difference between the desired size and the actual size to be offset when adjusting the instrument it is used with. Some digital instruments will allow you to program the zero setting function for this so all the user has to do is push the ‘zero’ button with the master in place and the offset will be accounted for and all readings will be variations from your desired size.

Some folks want masters made to closer tolerances than they need because they suspect the geometry of the master will be better and thus cut down on minor setting variations. While there is some truth to this there are no guarantees. The rule of thumb for plain ring or plug masters used to be that geometry would be held within fifty percent of the diameter tolerance. It would certainly hold true when comparing a ground master as opposed to one that has been lapped to bring it into tolerance but if the lapping is not done properly, the surface finish may be improved but not the overall geometry.

You should also keep in mind that the resolution of the instrument the master will be setting should be in keeping with the tolerance on the master. Overly sensitive instruments can drive inspectors crazy with dancing digits the magnitude of which may mean little in the grand scheme of things.

Our ability to measure masters to high orders of precision has limitations despite the tolerances often quoted in gage maker’s catalogs. This is due to measurement uncertainty of the calibration process being too high compared to the requested tolerance. This occurs most often with setting rings to XX tolerance which leads to battles over what the ‘size’ actually is when two different labs calibrate them. This should explain why if you ask a gage maker for a master with zero variation from the specified size, you may discover that your order will be declined. ‘Zero’ uncertainty is not achievable by any calibration laboratory including NIST.

If you have been imprisoned by some of the problems noted in this column you could always ask for a re-trial or parole to reset the situation and have your lawyer use this column when pleading your case. Take care and I’ll see you when you get out.

KEYWORDS: calibration cost of quality gages

Share This Story

Looking for a reprint of this article?
From high-res PDFs to custom plaques, order your copy today!

Hill Cox is the chairman for the technical committee for the American Measuring Tool Manufacturers Association and president of Frank Cox Metrology Ltd. He can be reached at [email protected] or at 905-457-9190. 

Recommended Content

JOIN TODAY
to unlock your recommendations.

Already have an account? Sign In

  • 2024 Quality Rookie of the Year Justin Wise 1440x750px banner with "Quality Rookie of the Year" logo inset

    Meet the 2024 Quality Rookie of the Year: Justin Wise

    Justin Wise is an exceptional individual who has been...
    Aerospace
    By: Michelle Bangert
  • Man with umbrella and coat stands outside while it rains at night looking at a building.

    Nondestructive Testing: Is there an ethics problem?

    I was a whistleblower who exposed fraudulent activities...
    NDT
    By: Dale Norwood
  • Unraveling Deflategate: Football stadium with closeup of football on field

    Unraveling the Tom Brady Deflategate

    The Deflategate scandal erupted following the 2014 AFC...
    Measurement
    By: Greg Cenker and Henry Zumbrun
Manage My Account
  • eMagazine Subscriptions
  • Newsletters
  • Online Registration
  • Subscription Customer Service
  • Manage My Preferences

More Videos

Sponsored Content

Sponsored Content is a special paid section where industry companies provide high quality, objective, non-commercial content around topics of interest to the Quality audience. All Sponsored Content is supplied by the advertising company and any opinions expressed in this article are those of the author and not necessarily reflect the views of Quality or its parent company, BNP Media. Interested in participating in our Sponsored Content section? Contact your local rep!

close
  • Key Takeaways for Quality Leaders
    Sponsored byComplianceQuest

    Key Takeaways for Quality Leaders from the 2026 Gartner Magic Quadrant™ for QMS

  • This image shows a person seated next to a Bobcat T66 compact track loader.
    Sponsored byPolyWorks by InnovMetric

    Supercharging Digital Gauging at Bobcat North America

  • Dorsey Calibration Lab photo by Tom LaBarbera Picture this Studios
    Sponsored byDorsey Metrology International

    Ensuring Product Quality in a Competitive Manufacturing Landscape

Popular Stories

a titanium diaphragm speaker driver

The One Thing Elon Gets Right Is Designed to Scare You

This image shows a person seated next to a Bobcat T66 compact track loader.

Supercharging Digital Gauging at Bobcat North America

Dorsey Calibration Lab photo by Tom LaBarbera Picture this Studios

Ensuring Product Quality in a Competitive Manufacturing Landscape

2026 Quality Professional of the Year!

Events

June 9, 2026

Future-Proof your Quality Processes with Advanced 3D Optical CMM Technology

Discover how to effortlessly capture complex data, leverage true multi-sensor automation, and ensure continuous operation without creating inspection delays.

June 22, 2026

Automate 2026

Automate is North America's largest robotics and automation event — and the best place to take your ideas from insight to impact.
 
Our show floor features the world’s leading automation solutions, from AI and robotics to motion control, vision systems, and more. Plus, our educational conference is second to none, led by the brightest minds in automation today.
 
Ready to transform the way you work? Take the next step at Automate.
View All Submit An Event

Products

Lean Manufacturing and Service Fundamentals, Applications, and Case Studies

Lean Manufacturing and Service Fundamentals, Applications, and Case Studies

See More Products
Quality Podcast Channel Custom Content

Related Articles

  • The Price of Precision

    See More
  • In Search of Precision Measurements

    See More
  • Zeiss Employee Working on Aerospace Metrology Solution

    Pushing the Limits of Precision

    See More

Related Products

See More Products
  • ZEuCDwAAQBAJ.jpg

    Lean Six Sigma In The Age Of Artificial Intelligence: Harnessing The Power Of The Fourth Industrial Revolution

  • Fundamentals of Plant Floor Layout DVD Package

See More Products

Related Directories

  • Masters Precision

    North American representative for moderate to high accuracy CMMs and Tomelleri Portable Arms, replacement (generic Renishaw CMM probing) providing, complete service, support and training for the products, as well as suppling Calibration and Software training for many major brands of CMMs.
×

Stay in the know with Quality’s comprehensive coverage of
the manufacturing and metrology industries.

Newsletters | Website | eMagazine

JOIN TODAY!
  • RESOURCES
    • Advertise
    • Contact Us
    • Directories
    • Manufacturing Division
    • Store
    • Want More
  • SIGN UP TODAY
    • Create Account
    • eMagazine
    • Newsletters
    • Customer Service
    • Manage Preferences
  • SERVICES
    • Marketing Services
    • Market Research
    • Reprints
    • List Rental
    • Survey/Respondent Access
  • STAY CONNECTED
    • LinkedIn
    • Facebook
    • YouTube
    • X (Twitter)
  • PRIVACY
    • PRIVACY POLICY
    • TERMS & CONDITIONS
    • DO NOT SELL MY PERSONAL INFORMATION
    • PRIVACY REQUEST
    • ACCESSIBILITY

Copyright ©2026. All Rights Reserved BNP Media, Inc. and BNP Media II, LLC.

Design, CMS, Hosting & Web Development :: ePublishing