This website requires certain cookies to work and uses other cookies to help you have the best experience. By visiting this website, certain cookies have already been set, which you may delete and block. By closing this message or continuing to use our site, you agree to the use of cookies. Visit our updated privacy and cookie policy to learn more.
This Website Uses Cookies
By closing this message or continuing to use our site, you agree to our cookie policy. Learn More
This website requires certain cookies to work and uses other cookies to help you have the best experience. By visiting this website, certain cookies have already been set, which you may delete and block. By closing this message or continuing to use our site, you agree to the use of cookies. Visit our updated privacy and cookie policy to learn more.
Subscribe
  • Sign In
  • Create Account
  • Sign Out
  • My Account
  • Home
  • The Magazine
    • Current Issue
    • Digital Edition
    • Archives
    • How To Guide
    • Industry News
    • Subscribe
  • Web Exclusives
    • Apps
    • Blogs
    • NDT Exclusives
    • Quality Exclusives
    • Vision & Sensors
  • New Products
  • Channels
    • Management
    • Measurement
    • NDT
    • Quality 101
    • Software
    • Test & Inspection
    • Vision & Sensors
  • Markets
    • Aerospace
    • Automotive
    • Electronics
    • Energy
    • Green Manufacturing
    • Medical
    • Plastics
  • More
    • Leadership 100 Survey
    • Plant of the Year
    • Professional of the Year
    • Job Board
    • White Papers
    • Quality Store
    • Software Downloads
    • eCards
    • Product Spotlights
    • Industry Links
    • Featured Products
    • Sponsor Insights
    • Market Research
  • Multimedia
    • eNewsletter
    • Podcasts
    • Videos
    • Webinars
    • Showcases
    • Image Galleries
    • Interactive Spotlights
  • Events
    • IMTS
    • Event Calendar
    • Quality Show
  • InfoCenters
    • Improving Manufacturing Quality
    • Manage Quality, Risk & Compliance
    • Materials Testing & Analysis
    • Modern Quality Control with SPC
  • Directories
    • Buyers Guide
    • NDT Sourcebook
    • V&S Sourcebook
    • Take a Tour
  • Contact
    • Contact Us
    • Advertise
Home » Photometric Stereo Technique
Vision & Sensors

Photometric Stereo Technique

Photometric stereo technique offers a quick, efficient way to use 3D data for better machine vision.

photometrics

Figures 1-4. In these photos, the black plastic clip is illuminated from linear miniature (LM) LED lights positioned at 90-, 180-, 270-, and 360-degrees around the tire’s perimeter and controlled by an LED Light Manager (LLM). As the camera triggers each exposure, the LLM triggers a light from a different direction. The camera feeds each image into a PC running an image library photometric stereo registration algorithm, which takes the best pixels from each image and combines them into one composition (figure 5) that reveals more than any constituent images. Photos courtesy of Matrox Imaging

September 4, 2018
Pierantonio Boriero and Matt Pinter
KEYWORDS 3D Inspection / 3D machine vision / 3D measurement technologies
Reprints
No Comments

Real-world objects have depth, width and height, and automated systems such as robots need to “see” in these three dimensions if they are to operate successfully. 

Machine vision systems composed of a camera attached to a computer running special image processing software give robots the “sight” they require. Ever since machine vision gained mainstream attention in the 1980s, however, one of the biggest challenges facing machine vision system designers has been how to best reduce the amount of data that needs to be processed to correctly locate, inspect and analyze an object. 

Data reduction meant that machine vision designers tried to find ways to employ lights, filters and cameras to solve color machine vision applications using black-and-white cameras. The resulting grayscale images contain less data, making them easier to process quickly. Similarly, engineers would develop mechanical fixtures, motion control systems and other methods to solve a traditionally 3D application—such as guiding a robot to pick an object from a moving conveyor—with a 2D machine vision solution. This 2D solution might have used mechanical fixtures, for example, to guarantee that “non-flat” objects were always presented the same way, eliminating the need for accurate height information for every pixel, in addition to width and depth location coordinates.

Today, microprocessors, graphic processor units (GPUs), field programmable gate arrays (FPGAs) and other computational engines give designers the luxury of more processing power—but processing power isn’t infinite as a quick review of modern 3D machine vision methods reveals. 

A Glance at 3D Vision

While the most challenging modern machine vision solutions that use technologies such as deep learning are being solved in the cloud, cost-effective processing power has relaxed the need for data reduction for the likes of color and 3D applications. For example, cheap data processing, lasers and optics have facilitated integrated laser triangulation systems for conveyor-based 3D systems that can generate tens of thousands of 2D profiles per second as a step toward creating a 3D object map. Other laser triangulation systems use galvanometers to quickly project a moving grid on stationary objects, resulting in high-resolution 3D surface maps. Conversely, new time of flight (ToF) cameras also offer low-resolution 3D maps for many applications without the safety concerns of laser illumination. 

For larger-area 3D projects, designers have several options. Single-camera photogrammetric systems mounted on the end of a robot are one example. These systems take multiple pictures of the same object from different locations and calculate the 3D position of every pixel in the image based on a predetermined geometric relationship between camera and object. In the case of large-area 3D inspections, developers can simultaneously align two cameras next to each other like human eyes to capture 3D information based on minute differences between two images of the same object caused by perspective shifts between the two cameras—again using the known relationship between the cameras instead of the camera and object.

But for high-speed inspection of objects where the field of view isn’t large, quantitative 3D data isn’t always required for measurement purposes, whereas qualitative 3D data can be very useful by itself. An emerging machine vision technique called photometric stereo can be the most cost-effective solution in these types of applications.

Photometric Stereo: Localized 3D

While photogrammetric systems return quantitative 3D data based on known geometric relationships, photometric stereo isn’t primarily concerned with measuring the height of any given pixel. Rather, the technique uses 3D surface orientation and its effect on reflected light to produce—among other possibilities—a contrast image accentuating local 3D surface variations, which are potentially invisible with traditional 2D imaging. Photometric stereo solutions do not need to know the exact 3D relationship between the camera and the object under test, nor do they require two cameras to capture 3D data. 

Instead, photometric stereo systems use a single camera with multiple sources of illumination. The photometric stereo technique for machine vision estimates the surface of an object by observing that object under different lighting conditions. This method is based on the fact that the amount of light reflected by a surface is dependent on the orientation of the surface in relation to the light source and the observer. 

Photometric stereo for industrial applications is gaining attention thanks to new specialized algorithms, a growing awareness of the importance of good lighting to machine vision success, and low-cost multi-light solutions that allow the control of four or eight lights through a simple browser-based interface at less than the cost of a frame grabber or smart camera break-out box. 

Today, the unique benefits of photometric stereo applications are enabling many common industrial inspection applications that once were difficult, if not impossible, to solve cost effectively. 

Clips and Tires

For example, reading raised letters on parts has always been problematic for machine vision systems, regardless of whether the part is a truck tire or an automotive clip. In this example, a plastic connector has several functional features on its surface as well as the number “2” and a directional symbol. As you can see from the constituent images, there is no difference between material that comprises the clip and the raised letter, hence there is no contrast. On larger objects, such as tires, manufacturers have used laser triangulation systems to create a 3D surface map. Laser scanning systems for 3D measurements have become much more integrated and effective, but are still costly solutions, often requiring the object to be moving during inspection, adding to the cost and complexity of the automated solution. 

photometrics

Figures 6-9. This set of four pictures shows synthetic leather material as illuminated individually from the four points of the compass. Visualizing 100% surface texture across the full image is nearly impossible for the human eye—much less a computer. The final composition from the photometric stereo registration algorithm (figure 10) shows an evenly illuminated texture across the camera’s full field of view with sharp contrast along each crevice and highlights perforations in the upper left quadrant. Photos courtesy of Matrox Imaging

In figures 1-4, the black plastic clip is illuminated from linear miniature (LM) LED lights positioned at 90-, 180-, 270-, and 360-degrees around the tires perimeter and controlled by an LED Light Manager (LLM). As the camera triggers each exposure, the LLM triggers a light from a different direction. The camera feeds each image into a PC running an image library photometric stereo registration algorithm, which combines all corresponding pixels to establish local surface properties—like curvature—and produce one or more types of composite images from these such as a contrast image of the local 3D geometries or an albedo image (figure 5). These composite images reveal more than any of the constituent images by themselves. The resulting composition clearly shows the edges that form the black-on-black lettering on the clips surface, as well as the edges of the various injection molded parts that make up the whole. 

Perforations and indentations

The next example shows four more pictures of a synthetic leather material (figures 6-9). Like the organic material that it mimics, leatherette has considerable surface texture. Visualizing 100% surface texture across the full image is nearly impossible for the human eye—much less a computer. 

In each constituent image, the warp of the material as it lies on a supporting substrate creates strong shadows while other parts of the image tend towards saturation due to strong light reflection. The final composition from the photometric stereo registration algorithm (figure 10) shows an evenly illuminated texture across the camera’s full field of view with sharp contrast along each crevice and the highlighting of holes. While organic materials like leather pose unique challenges for automated inspection systems due to constantly changing shapes and textures, the same can be said of pores on metal machined surfaces, such as engine heads, for example. 

Outlook and challenges

Direct part marking (DPM) systems such as dot peen, laser marking and even cast parts are another area that will clearly benefit from cost effective photometric stereo solutions. Combining the examples above with the knowledge that we do not live in a 2D theoretical world, we can safely say there are more 3D applications than 2D (or 1D) applications. This explains why DBMR Research recently predicted that the global 3D machine vision market will grow at a CAGR of 9.5% between 2017 to 2024, increasing from $15.4 billion to nearly $32 billion in annual sales. The primary challenges to 3D machine vision market growth include the high costs of installation and lack of technical knowledge. As shown by a packaged photometric stereo registration tool and one-click programming of the LLM light management system, the machine vision industry is quickly overcoming cost concerns and technical difficulties. V&S

subscribe to Quality Magazine

Recent Articles by Pierantonio Boriero

Deploying a Vision System

Vision Library or Vision-Specific IDE?

Machine Vision in Color

Accelerate Processing Performance

Pierantonio Boriero is product line manager at Matrox Imaging (Quebec, Canada). For more information, call (514) 685-7230 ext. 2757, e-mail Pierantonio.Boriero@matrox.com or visit www.matroximaging.com.

Related Articles

Machine Vision in Color

Accelerate Processing Performance

Vision Library or Vision-Specific IDE?

Deploying a Vision System

Related Events

The Quality Show 2019

You must login or register in order to post a comment.

Report Abusive Comment

Subscribe For Free!
  • Print & Digital Edition Subscriptions
  • eNewsletters
  • Online Registration
  • Subscription Customer Service
  • Mobile Help

More Videos

Sponsored Content

Sponsored Content is a special paid section where industry companies provide high quality, objective, non-commercial content around topics of interest to the Quality audience. All Sponsored Content is supplied by the advertising company. Interested in participating in our Sponsored Content section? Contact your local rep.

Close
Cncmachine__r0c0gpjprl__
Sponsored By
Renishaw Inc.

Renishaw and Hartford combine to deliver intelligent “smart factory” solutions

In the face of global skills shortages and rapidly emerging Industry 4.0 trends, Taiwanese CNC machine manufacturer Hartford undertook to develop an innovative, easier-to-use human machine interface (HMI) for its CNC machines. At the same time the company strived to ensure process measurement and inspection at its CNC machine manufacturing operations could keep pace with ever-improving product quality goals.

Popular Stories

JimGems

Positive Reinforcement

Ralco Industries

Quality 4.0: A Trend within a Trend

Quality Management

What is Quality Management and Why Does it Matter?

Renishaw Logo

Survival of the fittest – the process control imperative

ETQ

Top 3 Processes to Automate to Improve Quality Management

BuyersGuide


Quality Magazine how to directory guide video

Events

March 14, 2019

Why Excel is Failing Your Tolerance Stacks

This webinar will demonstrate a 1D, 2D and 3D stack analysis on a product, and explain why 1D and 2D stacks are missing the mark for tolerance analysis
March 20, 2019

Vision Guided Robots: 2D vs 3D Technology

This session will review 2D technology for inspection and guidance, and highlight how 3D vision is being used for recognition of randomly presented parts in bin picking applications.
View All Submit An Event

Poll

Topics to Talk About

What topics would you like to see Quality cover more?
View Results Poll Archive

Products

Quality Brain Teasers: Real World Challenges to Build Your Manufacturing Skills (ebook)

Quality Brain Teasers: Real World Challenges to Build Your Manufacturing Skills (ebook)

Assessments and Training in the following areas...

See More Products

SoftwareDownloads

Quality Magazine

Quality February 2019

2019 February

Check out the February 2019 edition of Quality: Quality Leadership 100 survey results, air gaging, Quality 4.0, automation and much more!
View More Subscribe
  • Resources
    • Market Research
    • Custom Content & Marketing Services
    • List Rental
    • Partners
    • Manufacturing Group
    • Polls
    • Privacy Policy
    • Survey and Sample
  • Want More
    • Connect

Copyright ©2019. All Rights Reserved BNP Media.

Design, CMS, Hosting & Web Development :: ePublishing