Quality Magazine logo
search
cart
facebook twitter linkedin youtube
  • Sign In
  • Create Account
  • Sign Out
  • My Account
Quality Magazine logo
  • NEWS
  • PRODUCTS
    • FEATURED PRODUCTS
    • SUBMIT YOUR PRODUCT
  • CHANNELS
    • AUTOMATION
    • MANAGEMENT
    • MEASUREMENT
    • NDT
    • QUALITY 101
    • SOFTWARE
    • TEST & INSPECTION
    • VISION & SENSORS
  • MARKETS
    • AEROSPACE
    • AUTOMOTIVE
    • ENERGY
    • GREEN MANUFACTURING
    • MEDICAL
  • MEDIA
    • A WORD ON QUALITY PUZZLE
    • EBOOK
    • PODCASTS
    • VIDEOS
    • WEBINARS
  • EVENTS
    • EVENT CALENDAR
    • IMTS
  • DIRECTORIES
    • BUYERS GUIDE >
      • Supplier Insights
    • NDT SOURCEBOOK
    • VISION & SENSORS
    • TAKE A TOUR
  • INFOCENTERS
    • Digital Quality Management Systems
    • NEXT GENERATION SPC & QUALITY ANALYTICS
  • AWARDS
    • ROOKIE OF THE YEAR
    • PLANT OF THE YEAR
    • PROFESSIONAL OF THE YEAR
  • MORE
    • Expert Columns
    • NEWSLETTERS
    • QUALITY STORE
    • INDUSTRY LINKS
    • SPONSOR INSIGHTS
  • EMAG
    • eMAGAZINE
    • ARCHIVES
    • CONTACT
    • ADVERTISE
  • SIGN UP!
Measurement

Measurement: Enemies of Precision Measurement

By George Schuetz
September 1, 2012
Watch out for these accuracy-attacking elements.

The Measurement Process

Source: Mahr Federal



When a gaging system is not performing as expected, the human tendency is to blame the gage or instrument for incorrect results. However, more times than not, the trouble lies elsewhere.

This is the natural reaction. People are quick to blame the instrument because it is easy to quantify. We can grab it, take it to the lab and test it. But this approach will often fail to find the problem, or may find only part of it, because the instrument is only a part of the total measuring system. Precision measurement is subject to many variables that need to be eliminated before we take the gage to task. The better approach is to step back and look at your measurement system as a whole.

A measurement system really consists of five elements: the environment in which the measurement is being conducted; the gage or measuring instrument; the part being measured; the measurement standard; and the person making the measurement. Now, there may also be issues with gage design that need to be accounted for and corrected, but more often than not, one or more of these elements will be the ultimate source of your problem. I like to think of them as enemies of precision measurement. In order to assure you are getting the best results from your measurement system you need to hunt down these enemies and eliminate them. Let’s look at some of the major culprits.



Obtaining Millionth Measurement

Source: Mahr Federal

The Environment

The environment might just be enemy number one as far as the measuring system is concerned. This is because the environment can have an influence on all the other parts of the measuring process. It will affect the standard, the workpiece, the people to some extent, and most importantly, the measuring instrument.

But we can’t think of the environment as a single enemy. It’s really a gang of thugs trying to steal microns from the gage’s accuracy capability. Who are the members of this gang? Temperature, dirt, humidity and vibration are the big four.

We know that virtually all materials change size with temperature. Temperature is a very difficult culprit to control, always moving and changing, coming through vents, windows, machines, people, light bulbs and so much more. Temperature is at its worst when it moves quickly. This throws the part, master and gage into utter confusion, resulting in what appears to be changing and unstable readings. To combat temperature variation, comparative gaging-that is, comparing the part to a master on a large mass gage made from the same material-can help. Then check the rest of the environment-air conditioning vents, sunlight from windows, parts hot off a machine-to keep big swings of temperature under control and minimize its influence.

Then there is humidity, a slow enemy that insidiously creeps into the process. You may not see it today, but give it enough time and you will start to notice the telltale signs it leaves behind. Gages can often “see” better than the human eye. Spots of rust caused by high humidity are measureable before they are visible. Rust on the reference standard makes for an erroneous zero setting, creating bias in all the part measurements. Soon, a whole batch of parts will have been offset and measured incorrectly.

Vibration is an enemy to very high accuracy gages, and especially to surface texture measurement. Vibration will appear as noise in the gage amplifier, or a seemingly rough surface when you know there is not one present. Or, vibration can find its way into the manufacturing process. Stamping machines send out lots of vibration that work its way through the machine tool and into the part. Eliminate vibration and get rid of these unexpected errors.

Dirt attains such preeminence as a common enemy because it, too, is everywhere. Dirt is so common we forget it and maybe even ignore it, but we can’t ignore its effects. As a fast experiment to observe how detrimental dirt can be to accurate measurement, leave your best digital micrometer out of its box for a few hours with the spindle turned back about a quarter of an inch. Then check it for zero setting. Next clean the anvils on a slip of clean paper in the customary manner and blow the lint away. Check the zero setting again. Don’t be surprised if you find a micron or so difference in the readings. You have just wiped out precision measurement’s number one enemy.

Speaking of dirt affecting the accuracy of gages, all you have to do to comprehend the “fairly clean” dirt errors mentioned above is recall the assortment of gages you often see in the grease and grit-laden areas around a machining center. The first thing a service technician does when starting on a repair job is to clean the gage, be it an indicator or the most intricate measuring system. Simple cleaning often is about all the “repairing” a gage needs.



Potential Looseness in Gages

Source: Mahr Federal



The Gage

Dirt is one of the causes of friction, a sticky little felon that might be classed as the number five enemy of accurate measurement. Acting primarily on the gage, friction takes its toll on gaging accuracy in several different ways, in addition to the impeding action of plain dirt.

Many mechanical gages have some type of transfer mechanism, whether it be a right angle transfer, transfer rod, lever arm, or ball on a Vee. Each one of these mechanisms can be affected by friction. In a transfer arm, for example, there usually is a pivot upon which the arm moves. If this pivot is adjusted too tightly it will not move smoothly and may possibly get hung up. Loosen it too much and it will be sloppy and cause non-repeat errors. Put a little oil or grease on this pivot and it becomes a magnet for dirt, and the combination of these two enemies is a certain disaster.

The opposite of friction, looseness is one of those back-to-basics gaging issues that often gets overlooked. But when a dial or digital indicator doesn’t repeat or calibrate properly, looseness in the contact points is often the cause.

While indicator contact points do need to be a bit more than finger tight, they should not be tightened up with pliers, or made so tight that the screw thread swells the end of the rack. Nevertheless, we frequently find the tolerances between the rack of an indicator and its bushing so tight that thread compression distorts the rack and causes it to swell. And here, of course, friction finds its way back into the picture.

But there are more connections in most gages than just the contact point on a stem. There are several familiar places on gages where looseness can be trouble. When an experienced troubleshooter works on a gaging system that isn’t coming up with the right measurement, he makes sure early in the game that all screws, nuts, bolts and connections are appropriately tight. One loose spring on a pantograph can be hard to detect, but the gage will know it’s there. Looseness can also be in the form of a single shoddy electrical connection in the electronic section or a loose and leaky air connection on an air gage.

Moving down the list of gage enemies is wear. Wear is difficult to deal with because it sneaks up on you over time and makes the gage unreliable before you realize it. When a conventional go/no-go plug gage wears, it becomes undersized and has to be thrown away unless it can be plated back up to size, or salvaged by regrinding it to a smaller size. Worn anvils on a conventional snap gage have to be lapped flat and parallel, and reset to size-a lengthy and costly process. However, under most circumstances, gage wear fails to bother the comparative gage user because the gage is checked with the master and re-zeroing usually compensates for any anvil wear.

But even here, caution is needed. In a case where the part and the master are the same shape this system works well-for example, where the master disc is used to zero a snap gage that is measuring a shaft. But if you use a rectangular master, like a gage block, and then gage cylindrical pieces, anvil wear can quickly become a factor.

The most common source of wear is where the contact point touches the part. Anvils or contact points on the indicators are obvious victims. But don’t forget possible wear points on the mechanical transfers in the gage itself. Every place there is a change of direction, or other transfer, wear can work its way in and cause issues.

Gaging pressure can be both friend and foe. If gaging pressure is right for the job, you come nearer to the correct reading. But if it’s wrong, it’s an enemy of precision measurement. Gaging pressure can also be a double-barreled enemy as it can affect both the gage and the part. So understanding this pitfall and accounting for it can minimize its influence. We are all familiar with what gaging pressure, plus the mechanical advantage of a screw thread, can do to the accuracy of a conventional micrometer reading, or even a strong thumb on a digital caliper. Too much gaging pressure can actually distort the gage and result in wrong readings. That’s why most hand tools now employ a ratcheting mechanism or thumb wheel to help reduce and maintain a constant gaging pressure.



Compressible Materials

Source: Mahr Federal

The Workpiece

Compressible materials such as plastics or nylons pose a different type of problem. Virtually any gaging pressure is going to distort the material. But if the industry knows this and clearly defines the characteristics relative to the gaging process, they can be standardized so that everyone measuring the material will get similar results. These standards include fixing the gaging pressure, contact shape, and size, along with the reference contacts.

You may not think of aluminum or steel as compressible materials, but make the part large enough and have thin walls, and you have a part that can get squeezed with the slightest gaging force. In cases like this you might have to consider ways to round up the part, or take multiple diameter readings to get the best estimate of true part size. This can be a tough enemy to defend against, although some newer electronic gaging systems are able to account for it.



The Standard

The terms magnification and resolution are often used interchangeably when discussing indicator devices. But in the world of dial indicators, magnification is used to describe things happening in the gear train, internal to the indicator, and does little to ensure accuracy. So while a big magnification number may make the purchase of a particular indicator more attractive, what most buyers really want is resolution. The idea of resolution comes from the relatively new digital products, and means, simply, the ability to distinguish clearly.

Take one inch on a steel rule as an example. The rule does not magnify the inch, but it does resolve it into divisions of 1/8-inch or 1/16-inch, or finer. You can even find scales out there where the resolution (the number of grads) is so fine that it virtually becomes useless because the eye cannot distinguish them. So choosing the right resolution for the gage is important for the operator to make his job easier, and give him the ability to distinguish good from bad and feel comfortable with everything in between.

Where resolution becomes an enemy is when the gage user thinks he can get better accuracy from a gage simply by changing the resolution of the indicator on the gage. For example, say you take a snap gage with a 0.0005-inch dial indicator and swap it out for a 0.00002-inch digital indicator. Certainly the resolution will be better in that you will be able to distinguish more gradations, but the gage will be no better able to measure to and perform to that level of accuracy. Parallelism of the anvils, friction in the mechanism, or looseness in the bearings will now become magnified. And what once were nice stable readings for the operator now become a series of flickering numbers running across the digital indicator. The operator either loses confidence in the gage or sends it out for unnecessary repair.

On the other hand, an indicator may not have enough resolution for an application. If the part tolerance is only a few counts on the indicator, then the flip of a single digit may represent a large portion of the tolerance and make a good part seem bad or a bad part look good. This enemy especially rears its ugly head when doing GR&R studies, where not having the right resolution for the tolerance-i.e., making the last digit flip-is an indication of bad measurement system repeatability.



Potential Hiding Places for Friction

Source: Mahr Federal



Other Causes

There are many more small-time hoodlums who are enemies to precision measurement. They could include iffy gage design, fixtures that are not robust enough for the requirements, choosing the wrong contact points for the application (flat on flat for example), or even the way operators make the measurement. Poor training, repetitive measurements, or an uncomfortable environment can make operators enemies of their own work.

The key is to know your enemies. Know their strengths and weaknesses. When you do, you will be able to conquer and destroy them, before they steal that micron-or dollar-from your back pocket. Q





Tech Tips

The environment can have an influence on all the other parts of the measuring process.

It will affect the standard, the workpiece, the people, and most importantly, the measuring instrument.

The environment is not as a single enemy. Temperature, dirt, humidity and vibration can all cause problems.



KEYWORDS: digital caliper gages manufacturing micrometers temperature testing

Share This Story

Looking for a reprint of this article?
From high-res PDFs to custom plaques, order your copy today!

George schuetz

George Schuetz has been employed with Mahr Inc. for over 40 years in the metrology business.  During that time, he has worked in the applications areas for Precision Gages, Special Gage Designs, Surface and Form Gaging working to solve customer's specific gaging problems.

Over the years, he has contributed articles to magazines including Quality in Manufacturing, Quality Digest, Quality Magazine, and is a monthly contributor to Modern Machine Shop. He has also written a book entitled Quality Gaging Tips, a compilation of modern machine shop tips.

Presently George is responsible for Precision Gage Product management at Mahr Inc. For more information, call (401) 784-3392, email [email protected] or [email protected] and visit www.mahr.com.

Recommended Content

JOIN TODAY
to unlock your recommendations.

Already have an account? Sign In

  • 2024 Quality Rookie of the Year Justin Wise 1440x750px banner with "Quality Rookie of the Year" logo inset

    Meet the 2024 Quality Rookie of the Year: Justin Wise

    Justin Wise is an exceptional individual who has been...
    Aerospace
    By: Michelle Bangert
  • Man with umbrella and coat stands outside while it rains at night looking at a building.

    Nondestructive Testing: Is there an ethics problem?

    I was a whistleblower who exposed fraudulent activities...
    NDT
    By: Dale Norwood
  • Unraveling Deflategate: Football stadium with closeup of football on field

    Unraveling the Tom Brady Deflategate

    The Deflategate scandal erupted following the 2014 AFC...
    Measurement
    By: Greg Cenker and Henry Zumbrun
Manage My Account
  • eMagazine Subscriptions
  • Newsletters
  • Online Registration
  • Subscription Customer Service
  • Manage My Preferences

More Videos

Sponsored Content

Sponsored Content is a special paid section where industry companies provide high quality, objective, non-commercial content around topics of interest to the Quality audience. All Sponsored Content is supplied by the advertising company and any opinions expressed in this article are those of the author and not necessarily reflect the views of Quality or its parent company, BNP Media. Interested in participating in our Sponsored Content section? Contact your local rep!

close
  • Key Takeaways for Quality Leaders
    Sponsored byComplianceQuest

    Key Takeaways for Quality Leaders from the 2026 Gartner Magic Quadrant™ for QMS

  • This image shows a person seated next to a Bobcat T66 compact track loader.
    Sponsored byPolyWorks by InnovMetric

    Supercharging Digital Gauging at Bobcat North America

  • Dorsey Calibration Lab photo by Tom LaBarbera Picture this Studios
    Sponsored byDorsey Metrology International

    Ensuring Product Quality in a Competitive Manufacturing Landscape

Popular Stories

This image shows a person seated next to a Bobcat T66 compact track loader.

Supercharging Digital Gauging at Bobcat North America

Dorsey Calibration Lab photo by Tom LaBarbera Picture this Studios

Ensuring Product Quality in a Competitive Manufacturing Landscape

a professional in the aviation field performing maintenance, repair, and overhaul (MRO) work

Manufacturing Retention: Strategies for Improving Company Culture, Engagement and Skill Development

2026 Quality Professional of the Year!

Events

June 22, 2026

Automate 2026

Automate is North America's largest robotics and automation event — and the best place to take your ideas from insight to impact.
 
Our show floor features the world’s leading automation solutions, from AI and robotics to motion control, vision systems, and more. Plus, our educational conference is second to none, led by the brightest minds in automation today.
 
Ready to transform the way you work? Take the next step at Automate.
July 14, 2026

Quality Leaders Forum: Better Communication, Better Quality Data

The Quality Leaders Forum is a quarterly, editor-moderated fireside chat series hosted by Quality Magazine, featuring candid conversations with senior manufacturing and operations executives shaping enterprise-level quality.

View All Submit An Event

Products

Lean Manufacturing and Service Fundamentals, Applications, and Case Studies

Lean Manufacturing and Service Fundamentals, Applications, and Case Studies

See More Products
Quality Podcast Channel Custom Content

Related Articles

  • Levels of Precision: A Field Guide to Dimensional Gages

    See More
  • Morehouse Digital Proving Ring with a white background.

    The Morehouse Proving Ring: A Century of Precision in Force Measurement

    See More
  • master ring gage being measured

    Benefits of Automating the Calibration Process

    See More

Related Products

See More Products
  • Measurement and Gaging DVD

  • Work Measurement DVD

  • principles of quality costs.jpg

    Principles of Quality Costs, Fourth Edition

See More Products
×

Stay in the know with Quality’s comprehensive coverage of
the manufacturing and metrology industries.

Newsletters | Website | eMagazine

JOIN TODAY!
  • RESOURCES
    • Advertise
    • Contact Us
    • Directories
    • Manufacturing Division
    • Store
    • Want More
  • SIGN UP TODAY
    • Create Account
    • eMagazine
    • Newsletters
    • Customer Service
    • Manage Preferences
  • SERVICES
    • Marketing Services
    • Market Research
    • Reprints
    • List Rental
    • Survey/Respondent Access
  • STAY CONNECTED
    • LinkedIn
    • Facebook
    • YouTube
    • X (Twitter)
  • PRIVACY
    • PRIVACY POLICY
    • TERMS & CONDITIONS
    • DO NOT SELL MY PERSONAL INFORMATION
    • PRIVACY REQUEST
    • ACCESSIBILITY

Copyright ©2026. All Rights Reserved BNP Media, Inc. and BNP Media II, LLC.

Design, CMS, Hosting & Web Development :: ePublishing