Smart Factory and Industry 4.0 are profoundly changing industrial value creation processes. They are characterized by a high degree of digitalization, networking and automation. All components involved are fully networked and communicate with each other via a wide variety of protocols. Innovations in robotics are also changing the face of industrial production. Small, compact and mobile robots are now dominating the scene in highly automated assembly halls. Collaborative robots (cobots) work closely together and share work steps with their human colleagues. They can also be quickly and flexibly retooled for different tasks.

Industrial image processing (machine vision) has become an integral part of this automated production scenario. Image acquisition devices such as cameras, scanners and 3D sensors seamlessly record the production processes. Integrated machine vision software then processes the image data and makes it available for numerous applications in the production chain. For example, the software can recognize a wide variety of objects based on optical features and position workpieces precisely. The technology also supports defect inspection. Defective products can be reliably identified and automatically sorted out. Machine vision monitors the entire production process, making processes safer and more efficient.

This applies in particular to the interplay between cobots and in interaction with humans. It is becoming increasingly important to optimize image processing algorithms for embedded platforms. The integration of both technology worlds is known as embedded vision. In the context of Industry 4.0, the use of compact devices with integrated embedded software is increasing significantly, especially smart cameras, vision sensors, smartphones, tablets and handhelds. The reason for their growing popularity in the industrial environment is that the devices are equipped with high-performance, industrial-grade processors that are available for the long term. This means that even complex image processing tasks can be carried out on them - provided they have powerful and robust machine vision software. In order for this to run smoothly, it must be compatible with a wide range of embedded platforms, such as the widely used Arm processor architecture. For example, the current release 18.11 of MVTec 's standard machine vision software Halcon can be operated on these platforms without any problems, both with 64 and 32 bit. Robust machine vision functions that otherwise only run on stationary PCs can also be used on compact devices.

Embedded vision systems can meet the high demands of digitalization, especially if they are equipped with artificial intelligence (AI). Such AI-based technologies include deep learning and convolutional neural networks (CNN). These methods enable very high and robust recognition rates. In deep learning processes, large amounts of image data are initially used to train a CNN. During this process, special features that are typical for the respective "class" are learned. Based on the training results, objects to be identified can be precisely categorized and recognized. The precise localization of objects and defects is also possible in the further course.

Deep learning in embedded vision applications

Deep learning functions are already used in many embedded vision applications. This typically involves large amounts of data. These are often non-industrial scenarios such as autonomous driving. Such vehicles have a large number of sensors and cameras that collect data on the current traffic situation. Integrated vision software uses deep learning algorithms to analyze the data streams in real time. This allows situations to be recognized and used for precise control of the vehicle. Deep learning-based embedded vision technologies are also used in the smart city environment. There, infrastructural processes such as road traffic, lighting or supply are digitally networked in order to offer residents a special service.

MVTec Halcon enables deep learning on embedded platforms. © MVTec Software

What are the advantages of using deep learning technologies in the embedded and machine vision environment? Deep learning algorithms are able to automatically learn specific distinguishing features such as texture, color or grey value gradient from the training data and weight them according to relevance. This task would otherwise have to be performed manually by image processing experts.

Object features are usually extremely complex and almost impossible for humans to interpret. If, on the other hand, differentiation criteria are learned automatically from training data, a great deal of effort, time and costs can be saved. Deep learning can also be used to distinguish more abstract objects, whereas traditional manual approaches only allow the classification of clearly describable objects. For example, objects with complex, intricate structures or objects in front of disturbing or very turbulent backgrounds can also be recognized. In most cases, a human would not be able to clearly distinguish between these objects.

As training requires very high computing power, complex neural networks are trained on suitably powerful PCs with high-end graphics processors. The trained network can then also be used on embedded devices, allowing compact and robust embedded vision solutions to benefit from the best possible recognition rates. AI-based technologies such as deep learning or CNNs are becoming increasingly important, especially in the highly automated Industry 4.0 environment. They are therefore already an essential component of modern machine vision solutions. If the algorithms also run on relevant embedded platforms, the full range of AI functions of robust image processing software can be used on compact devices.

Christoph Wagner, Product Manager Embedded Vision, MVTec Software / am