Visualization by the Viper system. © Rubedos Sistemos

The Viper system is a combination of 3D sensor technology and a flexible computing unit. It consists of a synchronized pair of commercially available 2D sensors, an embedded GPU computer platform integrated on the customer-specific carrier plate with all relevant industrial interfaces and carefully coordinated basic software. The latter was designed in such a way that it can be expanded using standard modules or customized Perception modules. These modules are also known as "Perception Apps", as they are available like in a smartphone app store and are easy to integrate. This software basis forms the most powerful function of Viper: it allows a captured 3D image to be additionally interpreted and qualified in the Viper system depending on the application. Only the image data relevant for the evaluation can be passed on to the host system. For example, the original 3D image can be analyzed and converted into the laser scan signal used by the robot. This converts the segment of the 3D image volume defined by the user into a 2D LiDAR signal representation almost in real time. Another use case is pallet space detection. Here, Viper can only output the relevant 3D coordinates of the pallet after it has been detected in front of a self-propelled forklift truck.

At the heart of Viper's 3D recognition is a passive stereo camera that captures synchronized images with a specific offset and uses fast on-board computing to reconstruct a 3D image of the surroundings as a point cloud. The stereo vision system with two 3.4 MP low-light cameras continuously measures the depth of the surroundings in 3D with a depth image rate of up to 40 Hz over distances of up to 50 meters. The reliable capture of the third dimension makes the 3D computer vision system immune to negative environmental factors that can usually affect 2D systems, such as a lack of lighting or contrast and the variable distance to the object of interest.

Viper first corrects image disparities using methods such as real-time lens correction, filtering, image equalization and other image error corrections, thus relieving the onboard processor of low- and high-level image processing tasks. In addition, there is sufficient power in the system to run local machine learning processes such as neural networks and process stereo image data. Viper uses such an approach, for example, to recognize people and calculate their relative 3D position in the image. This enables intelligent functions on the mobile robot platforms, such as a "follow the human" function. It can also be used to trigger an alarm if a human enters the predefined safety-critical zone, for example. To enable this functionality, the user activates one of Viper's extension modules, such as the Perception app "Follow-me". The app then combines the ability to recognize people (2D) and measure distance (3D) to inform the host system of any change in distance to the visually registered person. The mobile robot can thus slow down, brake or accelerate depending on the situation, thus enabling a "virtual distance line". In addition to tracking people, the app can also track other types of objects or special tags such as ArUco tags. The latter makes it possible to arrange a convoy of several UGVs (unmanned ground vehicles) that follow each other, while the foremost UGV follows a person.

Vision-based mobile robot platform. © Rubedos Sistemos

One of the key developments in Viper is the application of the Semi-Global Block Matching (SGBM) stereo matching algorithm optimized by Rubedos for the Nvidia Jetson family of embedded systems. Since speed is a crucial criterion in the application of vision-guided robotics, the configuration of the algorithms focuses on the speed aspect of stereo matching and treats accuracy as a secondary consideration wherever necessary. However, Viper allows to change the default configuration of the "speed" preference or to replace the preset stereo matching algorithm with a user-defined algorithm. This flexibility makes Viper an interesting stereoscopic vision tool for all robotics solution developers. As standard, Viper is available as a stand-alone, IP-protected and bolt-on system from "Eyes" and "Brain" for mobile robots. In addition, an OEM version is available for solution developers looking to seamlessly integrate Viper into their system design. With a weight of less than 250 grams and a power consumption of less than 10 watts, the OEM kit offers the same Perception features as the standalone version. However, the OEM kit is better suited for smaller UGVs and UAVs such as professional drones, which usually have a lower power and payload capacity.

With all these features, the Viper system enables intelligent automation with a turnkey set of hardware and embedded computing. It can be used in a variety of applications to equip industrial and service robotics with visual perception.

Linas Vaitulevičius, Rubedo Sistemos, and Andreas Breyer, Vision Communications / ag