Software and calculations are a recognized contributor in the uncertainty of the measured characteristic. Source: ISO/TS 14253-2:1999(E)

Using test data points in three dimensions, as shown for a sphere, simulates CMM measurements. Source: MicroEncoder

Measurements obtained with coordinate measuring machines (CMMs) are subject to several uncertainty components, much like any other measurement and test equipment. However, unlike most other such equipment, CMMs fit geometrical forms to the points gathered from various types of sensors. To calculate these geometrical forms, complex data fitting routines must analyze the data and fit the data to a particular geometry. Most CMMs are able to fit data to geometries such as lines, planes, circles, spheres, cylinders and cones using a least squares fit. In some instances, the least squares fit might not be sufficient; the software might be required to find minimum circumscribed, maximum inscribed and minimum zone data fits as well.

History has shown that the data fitting has been a source of measurement uncertainty. Issued in 1988, the Government-Industry Data Exchange Program (GIDEP) alert X1-A-88-01 informed CMM users that CMMs were found to indicate different results for the same dimension and measurement technique using different least squares algorithms. Only a few years earlier, these errors were thought to be negligible. While improvements have been made for least squares fitting software over the past decade, more complex algorithms, such as the minimum circumscribed, maximum inscribed and minimum zone fits, still has a strong potential to provide inaccurate answers. More recently, a 2002 study conducted by the National Institute of Standards & Technology (NIST) found that for the minimum circumscribed, maximum inscribed and minimum zone fits, serious problems can exist with commercial software packages.

Fortunately for CMM users, the GIDEP alert initiated action by NIST to create the Algorithm Testing and Evaluation Program for Coordinate Measuring Systems (ATEP-CMS) to allow for testing of least squares fitting software. This work, in cooperation with the American Society of Mechan-ical Engineers (ASME), has led to the creation of the national standard ASME B89.4.10-2000. The standard contains data set descriptions and provides a framework for testing CMM least squares fits for lines, planes, circles, spheres, cylinders and cones.

ISO 10360-6: 2001(E) also provides a framework for testing that is somewhat different than ASME B89.4.10-2000. Fortunately, work is underway to harmonize the standards to make the tests as similar as possible between the standards for customers and manufacturers. While the ATEP-CMS tests least squares fitting algorithms, data sets and reference fits have been generated for other types of fits as well.

A partial scan of a NIST report contains software performance values associated with the test data compared to expected values for specific conditions. Source: NIST

Obtaining information on uncertainties

What steps can be taken to gain information about measurement uncertainty due to CMM software? For most CMM users, a good first step is to contact the CMM manufacturer for information on how, and if, the software's contribution to measurement uncertainty has been evaluated for the software. Many companies have used the ATEP-CMS service from NIST and may be able to provide a copy of the test report indicating the uncertainties derived from deviations from NIST reference fits. One important concept to remember is that the test report covers only the conditions stated; the test data sets do not cover all possible types of data sets that could be obtained when measuring. Other countries provide some testing services as well. Because of the cost, CMM owners usually do not undergo NIST's ATEP-CMS test service on their own.

If unable to obtain a NIST test report, there are still some straightforward means of investigating the software's performance.

• Data sets and reference fits may be available-for instance, from NIST-for self checking. This differs from the ATEP-CMS test service in that it is self-performed, and a test certificate is not produced. NIST has some reference fits available for least squares, minimum circumscribed, maximum inscribed and minimum zone fits.

• When reliable reference fits are not available, fit results can be compared when the same data sets are submitted to a second software package that uses different fitting algorithms. While this method has limitations because of unknown reliability in both software packages, there still may be benefits, particularly as an alert to problems when results from the two packages differ.

• Using calibrated artifacts can be an indirect reference result for testing the software. For example, consider measuring the diameter of a calibrated cylinder using a CMM. Software fitting errors would affect the difference between the measured and calibrated values. Using this method, the software errors are not isolated from the hardware errors. Even so, serious software errors could be detected. This method is limited because of the normally small number of calibrated artifacts available.

• Another quick but limited method of evaluating software performance relies on simple, special cases. It may be possible to generate data sets with known solutions in certain cases. For example, three points forming an equilateral triangle can be selected on a circle. Additional points can be added slightly outside the circle. Then these points taken together form a data set whose maximum inscribed circle is known to be the original circle.

• Intuition and pictures can provide some insight into software performance. Sometimes intuition allows for creating instances that are problematic to software, such as measuring cones with very large or small apex angles, or measuring points that lie on a small arc of a circle. Pictures of the data set and reported solution can be helpful, as sometimes errors are large and can be revealed immediately when viewed graphically.

CMM measurement uncertainty because of software has been problematic in the past, and has the ability to be a continued source of uncertainty, especially for minimum circumscribed, maximum inscribed and minimum zone data fits. Many CMM manufacturers test their software using NIST data sets for least squares fitting, which has led to improvements in least squares fitting software over the past decade. National and international standards provide a framework for testing the CMM geometries. If needed, other limited scope methods are available for CMM users to test their software. These methods provide simple tests and instant results.

Disclaimer: Commercial equipment and materials are identified in order to adequately specify certain procedures. In no case does such identification imply recommendation or endorsement by the National Institute of Standards and Technology, nor does it imply that the materials or equipment identified are necessarily the best available for the purpose.

Many CMM manufacturers evaluate their software using NIST test data sets. Usually multiple-sensor CMMs use the same algorithms for fitting. Photo: Mitutoyo America Corp.

Tech tips

• CMMs were found to indicate different results for the same dimension and measurement technique using different least squares algorithms.

• NIST created data sets representing geometries for CMM manufacturers to verify their software with the least squares fit.

• ASME B89.4.10-2000 defines the tests and provides a framework for testing CMM least squares fit for planes, cylinders, cones, circles and lines.

• Still other methods can determine whether the software is a significant source of uncertainty.

• National and international standards provide a framework for testing CMM geometries.