Ever since the first robotic systems were developed, researchers have been looking at how they can be programmed quickly and intuitively. The hypothesis is that robot systems will become more profitable and therefore more widespread in small and medium-sized companies and households the more intuitively they can be programmed. The focus here is on the fact that the robot systems must be able to be operated and programmed by non-experts who have no knowledge of a programming language.

Various approaches have already been developed to achieve this goal. In the area of "programming by demonstration", for example, the robot is programmed with the help of several demonstrations of the task to be performed. Another approach is programming through virtual or augmented reality (VR/AR), in which the user programs the robot system using VR/AR glasses and pointing gestures. The final approach is playback robot programming, in which the user takes the robot by the hand and guides it through the desired task, with a programming system recording the demonstrated execution in the background. The recorded task execution is then played back identically by the robot.

The Chair of Robotics and Embedded Systems at the University of Bayreuth takes the concept of playback programming and extends it with a graphical user interface. The user interface is characterized by the fact that it uses elements of video editing and allows the recorded robot programs to be edited. The graphical structure is based on the typical structure of video editing software, in which individual timelines for different robot (components) of the system, a 3D preview for the simulation of robot movements and a toolbar for various functions are provided. The central component of the programming system is the translation of the hand-guided robot program into a graphical representation along timelines, which should be as easy to understand as possible. On this representation, the recorded movement can be edited and extended before a program is executed, so that very complex robot programs can be created from the original recording.

A screenshot of the graphical user interface with two robots. © University of Bayreuth

Subsequent editing and integration of sensors
The special features of the system include the ability to edit the recorded programs at a later date and the simple integration of external sensors in order to react to changes in the environment. By displaying the robot programs along timelines, it is possible to edit them using drag-and-drop with the typical editing options of video editing (copy, paste or delete). This means that parts of the recorded movements can be repeated and errors can be removed from the program without having to re-record the entire movement.

In addition, users have the option of inserting sensor impressions in the form of camera images, for example, into the robot programs and evaluating them during execution. With the help of the sensor system, it is possible to incorporate repetitions or alternatives into the programs based on the environmental conditions detected. In the case of repetitions, the sensor impression is used to decide whether a part of a program should be executed a second time, while an alternative is used to decide which part of the program should be executed. This functionality is based on the concepts of the loop and the conditional statement, which can also be found as control structures in classic programming.

Possible applications
The fields of application of this programming concept are diverse and can be used particularly in the area of small series production. The operation, which is geared towards non-experts, enables new programs to be created quickly, which is necessary even for small changes in the environment. At the same time, manual guidance for processes that require special skills, such as surface coating, allows the user's experience to be directly integrated into the programming process. One possible application is the assembly of components or the sorting of workpieces, for which an industrial robot with a gripper is used. During sorting, the sensor impressions can then be used to distinguish between different workpieces. Another possible application is the application of material to workpieces. In the case of fiber spraying processes, for example, a fiber-slurry mixture is applied to a raw mold. In addition to the robot, other devices must be programmed for such a process, which are also integrated into the user interface.

There are currently two prototypes that are used in the above-mentioned applications. In the future, it is planned to expand the user interface with additional functions, such as manual adjustment of movement within the 3D simulation, and to make the operation of the new functions intuitive for non-experts. as

© MHI

Briefly explained: The MHI e.V.
The Wissenschaftliche Gesellschaft für Montage, Handhabung und Industrierobotik e.V. (MHI e.V.) is a network of renowned university professors - institute directors and chair holders - from German-speaking countries. The members conduct both fundamental and application-oriented research on a wide range of current topics in the fields of assembly, handling and industrial robotics. Further information on the society, its members and activities: http://www.wgmhi.de.

© University of Bayreuth

Briefly explained: Robotics at the University of Bayreuth
The Chair of Robotics and Embedded Systems at the University of Bayreuth was founded in 2003 by Prof. Dr. Dominik Henrich. It deals with robots as information-processing systems that can detect, change and interact with their environment. One focus of research is the coexistence and cooperation of humans and robots. The aim is to eliminate the strict spatial separation between humans and robots in order to combine their strengths synergistically. Other current focal points are intuitive robot programming and instruction and CAD reconstruction with hand-held depth cameras. www.ai3.uni-bayreuth.de.