Further developments in robotics and image data analysis are helping to continuously expand the range of applications for collaborative assembly. This offers scope for the development of new concepts to further increase the intuitiveness and safety of assembly operations in human-robot collaboration (HRC). One promising approach here is to track the positions of humans and robots in order to detect potential collisions at an early stage.

By responding to the presence and movements of human hands, the robot improves safety in the working environment and increases efficiency at the same time. The development of such concepts marks a further step towards a seamless and safe coexistence of humans and robots in industry. Although such collisions in HRC are covered by force and power limitations, a proactive reaction of the robot to the presence of the human could prevent or at least mitigate these collisions.

As part of research at the Bremen Institute of Structural Mechanics and Production Systems, a concept is being developed to improve human-machine interaction within collaborative assembly scenarios using image-based hand recognition. The aim is to adapt the reaction of the robot used depending on the position, speed and acceleration of the hand. In addition to current image data, past information and conditions are also taken into account in order to be able to react appropriately to the situation, for example if the hand is shaded from the camera. This approach expands the interaction possibilities in HRC and reinforces the existing safety technology.

Different hand detection algorithms were compared for implementation, with the Single Shot Detector being identified as the most efficient choice. The concept for speed control is based on the EN ISO 13855 standard in order to define a safety zone around the robot's TCP and to develop distance controls. Latency and inaccuracies of the robot and camera are taken into account. Based on a coordinate transformation, the detected hand positions are translated into the robot's coordinate system.

A key problem with the chosen approach of observing the workspace from above is the potential obscuring of hands by the robot. An algorithm was therefore developed that recognizes the number of hands in the work zone, stores their position and stops the robot if necessary based on a case distinction. If, for example, one or more hands are in the robot's work zone, the robot only resumes full speed when the hands are detected outside the work area. This decoupling of hand detection in real time and the last detected position within the work area means that an unintentional reaction if a hand is not detected can be ruled out.

Future work can further expand the extended interaction possibilities in HRC. One possible optimization would be the implementation of an approach in which the robot actively avoids the hands in order to completely prevent potential collisions. Furthermore, this approach could also be used to train a robot to follow the hand movements of a human along a predefined movement path with waypoints. The integration of gesture control could further intensify the interaction between humans and robots in HRC.

Briefly explained: the bime

The Bremen Institute of Structural Mechanics and Production Systems (bime) is an institute of the University of Bremen in the Faculty of Production Engineering - Mechanical and Process Engineering. The bime currently employs around 70 people. The Assembly Systems and Production Design departments, headed by Prof. Dr.-Ing. Kirsten Tracht, develop solution concepts for the production of complex products in increasingly networked and agile environments. In addition to the development and operation of production systems, the design of assembly processes, the layout of handling devices and the modularization of production systems and equipment are the subject of the research work. The working group also focuses on planning tools for efficient planning processes and high-quality planning results that can be achieved at an early stage. The bime, represented by Prof. Dr.-Ing. Kirsten Tracht, is a member of MHI e.V.(http://www.bime.de)

Briefly explained: the MHI e.V.

The Wissenschaftliche Gesellschaft für Montage, Handhabung und Industrierobotik e.V. (MHI e.V.) is a network of renowned university professors - institute directors and chair holders - from German-speaking countries. The members conduct both fundamental and application-oriented research on a wide range of current topics in the fields of assembly, handling and industrial robotics. Further information on the society, its members and activities: http://www.wgmhi.de