Of all the measurements taken on metal parts, from conveyorized production-line items to specialized, one-of-a kind parts, a company's greatest concern stems from a surface roughness measurement that has varied by a millionth of an inch.

To be sure, concern can be justified for fuel injectors and other ultra tight-tolerance parts. If the scale and proportion of the part is such that the total surface roughness allowance is 10 parts of a million, then 1 part-of-a-million variance is significant. At the same time, it is easy to forget that 10 parts of a million is still microscopic, a fact that is somewhat obscured by using measurement terms such as micro, Ra or roughness.

Still, miniscule differences in measurements can be addressed, and calibration methods and techniques are available to help achieve better accuracy and uncertainty levels.

Calibration is a comparison of any measuring instrument with a known standard; the quintessential example is the use of gage blocks to calibrate measurements of length. Surface roughness is a dynamic measurement, not a static one, and the condition of the individual elements that collectively capture the data can contribute to the result. These elements -- the stylus, pick up and instrument -- must be considered while calibrating.

Equipment manufacturers have simplified the calibration process. Most new equipment allows the operator to measure a known standard, compare the instrument result to the stated value, and adjust a potentiometer until the measured result and the standard value are equal. More sophisticated systems feature computers that allow an operator to input or download the standard value, which is then compared to the measured result. Deviations are electronically compensated so that subsequent measurements will be correct. The procedure is easy; the computer crunches most of the data and computes the information in complex formulas. Still, mistakes and inaccurate results are possible because of the elements that contribute to a surface roughness measurement.

The stylus. The stylus traces, or maps, deviations in the machined surface. Surface characteristics dictate what tip size and geometry to use. Diamond stylus tips are generally conical in shape with either a 60 or 90 degree included angle. The tip radius is usually 2, 5 or 10 microns. If the stylus tip is worn, damaged, dirty or incorrectly identified to the instrument, the results will be wrong. The instrument or computer can't help because it cannot tell the condition of a stylus tip.

The pick up. The surface deviations traced by the stylus must be converted to a digital value by means of an electro-mechanical device called a pick up. In most cases, the stylus is attached to a lever arm that moves in an inductive field to generate an electrical signal. Much like any other transducer or gage head, the key elements are range, resolution and linearity. Range being the maximum vertical displacement, resolution being the minimum vertical step and linearity being the likelihood that a vertical movement anywhere along the total range produces an identical electrical response or signal. If the pick up is damaged, worn or dirty, it cannot be relied on to correctly reproduce the movement of the stylus throughout the measuring range.

The instrument. Surface deviations are extremely small. To better examine the data, the recording instrument must amplify, or magnify, the signal received from the pick up. The data is transformed and enlarged into a graph that makes a 3-inch mountain out of a millionth-of-an-inch molehill. An adjustment to the data, called a gain, is required to ensure that the signal is amplified correctly. Early generations of surface measuring instruments required a fair amount of tweaking of several different components during a typical workday to maintain accurate results. Modern instruments typically have only one "tweaker," a simple potentiometer that adjusts the amplitude gain. Some PC-based instruments only need to be programmed once with the correct configuration information and require no further tweaking.

The calibration specimen
A complete mechanical and electrical calibration of all functions and characteristics for a roughness measuring system involves a lengthy and costly procedure. An accepted practice is to calibrate using a limited number of checks and specimens. The assumption is that if an instrument passes within the scope of one check, it passes throughout the entire range.

The most prevalent specimen is the "Ra patch" or roughness standard. It is typically an electroformed replica of a precision-machined roughness specimen with uniformly spaced grooves having uniform amplitude. The grooves may be sinusoidal, triangular or triangular with truncated peaks and valleys. Groove spacing and amplitude are generally designed to produce a Ra value of approximately 120 microinches or 3 microns. If an instrument reads, or can be adjusted to read the specimen's Ra value correctly, then assume that the surface measuring system is operating correctly. However, it is not uncommon for an instrument to pass this test but fail to correctly measure a machined surface or correlate with another instrument measuring the same surface. Several possibilities for this error exist.

The Ra patch has widely spaced grooves and a large included angle between grooves that can accommodate a stylus that was damaged or has worn beyond its useful life.

When that same stylus is used to measure a machined surface with a finer texture, it cannot trace and reproduce the surface deviations correctly. Periodic checks of the stylus' condition are critical. Several calibration standards are available that have two patches; one is rough and is used for checking Ra, and the other is smooth for checking the stylus' condition. The smoother patch has closely spaced grooves and an included angle just slightly greater than 90 degrees. If the stylus is excessively worn or otherwise damaged, it will not trace this surface correctly and the resultant Ra value will be less than indicated on the standard.

A common mistake is to adjust the instrument to read the stylus check patch value. This oversight can result in significant errors. The correct procedure is to measure the smooth patch and record the value. Then measure the rough patch and adjust the instrument as necessary to attain the value indicated on the standard. Next, measure the smooth patch again. If the measured value is within acceptable limits, in the range of I5%, assume that the stylus is in good condition. If the measured value is beyond acceptable limits, the stylus should be replaced. Although some approximation can be made about the condition of the stylus based on the Ra value, this stylus check is best considered as a go-no go gage.

A different sort of standard can be used to better quantify the stylus tip size, angle and condition. This standard has a number of straight-sided grooves engraved into a piece of glass. Each groove has a different width and corresponds to one of the typical stylus configurations, 1, 2, 5 or 10 micron radius with 60 or 90 degree included angle. A measurement across the grooves is carried out and the resultant graph will exhibit to what extent the stylus penetrated each of the grooves and determine the size and condition of the stylus. This is not a shop-floor calibration method and it should be conducted only under laboratory conditions.

Errors derived from pick up
Another source for instrument error is incorrect amplification of the electrical signal derived from the pick up. Although the Ra patch makes use of that signal for a calculation of average roughness height, it cannot isolate vertical displacement of the stylus to a specific unit of length. For that purpose a step-height master is used. This master has a precise groove engraved across a block of glass; the depth of the groove is calibrated and noted on the specimen. The procedure is to traverse the stylus across the groove and then measure the mean depth of the groove relative to the mean level of the surface outside of the groove. Mean is emphasized here to caution against the use of total height parameters Rt and Pt to evaluate the depth of the groove. Both parameters can be affected by inherent roughness, however slight, on the measured surfaces. If the measured step height differs greatly from the calibration value or if the correct value can be derived only within particular portions of the total gage range, it is likely that the pick up has been damaged by mishandling or excessive wear. Although it is often possible to find a "sweet spot" and complete the calibration, proceed with caution and plan to replace or repair the pick up as soon as possible.

Surface finish measuring systems are available that can calibrate a spherical surface, typically a highly polished tungsten carbide ball. The principle advantage is that all elements of the system can be simultaneously checked and calibrated. Linearity of the entire gage range is verified because of the amount of vertical travel required to traverse the ball. The size, shape and condition of the stylus tip are checked because all of the points along the tip are contacted during traverse across the ball. Finally, all of the instrument software functions, including form removal, can be monitored in the course of calculating the result. Because calibration constants such as ball radius are programmed into the instrument and then removed from the measurement, the ideal result is a perfectly straight line. To whatever extent and in what manner the actual profile deviates from a straight line is an indication of overall system integrity. If the deviation is within proscribed limits, then compensating factors are applied and the calibration is accepted.

If the deviation exceeds those limits, the calibration is refused and individual-operating elements must be examined as described above.

When either the step height or the ball calibration procedure is carried out correctly, there is no need to also calibrate using a Ra patch. However, many shops routinely calibrate with the step height standard and then measure the Ra patch as a means of verifying overall system operation or to comply with internal quality conformance procedures. If the Ra value is not within acceptable limits, typically + or - 5%, then something is wrong. Check the condition of the Ra patch for excessive scratching or contamination in the grooves. If available, try another Ra patch for the system to check and compare the values. If the step-height method was used, check the condition of the diamond stylus tip.

Most important, don't tweak anything on the instrument that isn't recommended in the manufacturer's operating instructions because current generation instruments are rarely to blame for failure to calibrate correctly. By following this and a few more practical tips, that millionth of an inch variance that causes such consternation among manufacturers can be eliminated or reduced and measurement uncertainty can be minimized.