For quality professionals, the modern 3D smart sensor has moved to the center of conversations around quality in the automated age. Though 2D imaging remains popular, the rising affordability and strategic advantages of 3D vision for the smart factory are difficult to overstate. The future points toward increased use of robotics, machine learning, and real-time data on the shop floor, all of which frequently rely on 3D smart sensors.

For example, many production lines today have a robot for assembly and, further down the line, a machine vision system for inspection. But what if the robot had 3D vision built in, and thus could detect and correct problems at the front of the line? Already a reality, this mode of smart, automated, and data-rich quality inspection is on pace to become the new normal. 

Terry Arden, the CEO of LMI Technologies, is one notable contributor to this evolution. In 2009, Arden invented LMI’s Gocator product line: an all-in-one 3D smart sensor solution focused on delivering inline inspection for 100 percent quality control in smart factories. Fueled by a passion for building “smartware”—technology with the ability to see, think, and do—Arden is at the forefront of a new movement to incorporate deep learning into vision technologies. 

In an email exchange, Arden describes how 3D smart sensors are transforming the vision market. He also explains how 3D vision technology can unlock new possibilities for inspection, measurement, and quality control.

Quality: How are smart 3D vision sensors getting smarter?

Terry Arden: 3D smart sensors continue to evolve as production processes change and quality control tolerances get tighter. Onboard software is a critical vehicle for this evolution that includes adding measurement tools to meet specific application demands, supporting multi-sensor alignment to allow customers to see more of their part, and launching easy robot integration to provide customers with vision-guidance and flexible inspection capabilities in emerging robotic applications (e.g., bin picking, pick and place).

In what ways are advancements in machine learning and other aspects of Industry 4.0 affecting the development of 3D sensors for inspection and automated quality control?

This is really a question of improving data understanding, and it is a huge topic in machine vision today. While smart sensors already have the advantage of being able to process raw image data directly onboard the sensor, the emergence of new machine learning algorithms (with their much greater memory/processing demands) is driving a change in smart sensor design—mainly figuring out how to fuse deep learning (DL) model execution with the classic compact, all-in-one smart sensor design for 3D inspection.

LMI has addressed this need to add machine learning to a 3D smart sensor solution through a dedicated GPU-driven data processing hardware device (GoMax) that we call a vision accelerator. Such an accelerator has the necessary compute power to execute DL models without the need for industrial PCs.

The industry is recognizing that accelerators are needed to assist existing data processing pipelines in smart sensors by leveraging advanced GPU-processing, and this is leading to a network of devices sharing the overall inspection processing load to deliver results at factory speeds. This distributed architecture is driving the design and development of our smart sensor platform.

Smart 3D vision is critical to robotic applications. Source: LMI Technologies

In this sense, 3D smart sensors are evolving to support both on-board, edge-based computing and the IIoT (connecting to the factory network with accelerators and robots in close proximity).

How is measurement accuracy improving?

The willingness and commitment of sensor manufacturers to develop custom components (camera chips, optics, etc.) is the driving force behind higher resolutions and increased measurement accuracy. These higher resolutions allow for more accurate measurement at inspection speeds that simply could not be achieved before. For example, our latest Gocator 2500 series laser profiler sensors produce 3D scans at up to 10kHz. This was achieved by investments in a custom camera chip design and specific optics to maximize light gathered into such a high-speed camera.

What are the challenges to growth for 3D vision sensors and embedded vision today?

There is still a widely-held industry perception that 2D is inexpensive and easy to use, while 3D is complicated and costly. The challenge for 3D sensor manufacturers is therefore to educate engineers on the benefits of 3D, as many of them still don’t know that 3D is a more complete solution (you measure both shape and contrast), easy to set up (sensors are pre-calibrated with built-in light and optics), and cost-effective (easy to deploy, no on-site calibration, and ready to measure at a similar price point [to] a full 2D inspection solution that doesn’t measure shape).

Getting this message across is the biggest obstacle to achieving a major market transition from 2D to 3D. Education includes developing marketing content and training programs that show the benefits of 3D over 2D and slowly transforming integrator and developer capabilities by achieving first successes in a factory using 3D.

Applications like glue bead track inspection can only be solved with 3D. Source: LMI Technologies

Finally, what are the top trends you’re seeing in this space?

In order to meet the demands of emerging quality control applications, the industry is undergoing a fundamental transition from 2D to 3D technology. There are many applications where 3D is the only viable solution for automated quality control. As the level of automation increases and quality becomes more critical (given the volumes and cost of goods in industries like automotive or consumer electronics), it means there is a growing opportunity toward more and more production processes that use 3D.

For example, 2D technology cannot be used for adhesive and sealant inspection because 2D vision is unable to distinguish transparent glues from the surrounding material. In addition, 2D is unable to measure height data and volume. 3D smart sensors on the other hand are capable of providing high-resolution height data, which allows users to accurately determine glue bead location and volume.

Another application is battery inspection, which in general is a critical process both from a safety and a reliability. Once again, 2D does not measure flatness and cannot perform the inspection required to ensure proper battery assembly.

3D is also a natural solution over 2D in robotic applications such as bin-picking. Manufacturing is moving rapidly into these type of robotic applications, further driving the need for robust 3D solutions as the global industrial world moves toward increased automation with robots. V&S