Voice control of robots has been present in many facets of research for over 20 years now. For a long time, however, user-friendly use outside the ivory tower was not possible due to a lack of robust speech recognition. Thanks to immense progress, however, robust voice control has now been possible for several years and is already being used in smartphones, navigation systems and households. However, natural language control in conjunction with a dialog system offers advantages not only for households, but also in industry and the SME sector. The worker does not have to familiarize himself with a programming language first, does not need complete knowledge of the robot's capabilities and can carry out other activities during the instruction.

Current industry-related work in the field of speech processing covers a number of areas relevant to industry, such as intralogistics using voice instructions for mobile robots or the assembly of components using voice control of robot arms. However, most approaches deal either with speech recognition or the possibilities of defining robot movements, but not explicitly with the parameterization of force-based movements using the information contained in the instructions.

Force-based here means that the robot arm must either maintain a certain force during movement (force-guided) or must not exceed a specified force (force-monitored). The DFG project VerbBot (Verbal Instruction of Sensor-based Robotic Systems) of the Chair of Robotics and Embedded Systems at the University of Bayreuth is investigating how to close this gap and enable flexible natural language instruction of robots for force-based tasks. Among other things, a speech-based dialog system is used, which supports the worker in solving problems in the event of incorrect instructions.

The VerbBot framework extracts parameters from an instruction and converts them into a robot movement if they are valid (right path). If the instruction is not valid, feedback is generated (left path). © University of Bayreuth

Flexible control is made possible in the VerbBot project by automatically extracting the parameters required for the robot's movements from natural language instructions and, if necessary, pointing out errors in the instructions to the worker. The aim is therefore to be able to instruct the system like a human expert and to receive assistance if incorrect, incomplete or ambiguous instructions are recognized. Even non-experts in the field of robotics should be able to instruct force-based robot movements without causing damage in the event of an error.

On the same wavelength
For non-experts in robotics in particular, it can be difficult to assess how a robot will perform a movement. To get a handle on this problem, elementary robot movements (pushing, pressing, turning) are defined based on physical laws, as these laws apply to both the worker and the robot.

Combinations of these elementary movements can then be used to define any other movements or skills (screwing, cutting, wiping). These skills, here called Combined Verbalized Effects (KVE), are initialized at runtime based on material data from an object database and sensor data. For example, it is sufficient to say: "Clean the work area with the sponge" instead of giving the robot the joint and force data. The system also offers the option of using fuzzy parameters, such as light or firm, to adapt the movements.

Itcosts nothing to ask
Even if a system is available that enables all the necessary movements to be instructed, this does not mean that they will be transferred without errors. It can happen that a movement cannot be executed due to a lack of sufficient knowledge about the robot's capabilities, perception or necessary parameters. Such bottlenecks are solved in VerbBot using a voice-based dialog system. Once an instruction has been recorded, it is checked for physical feasibility (e.g. maximum joint angle, collision) and sufficient and unambiguous parameterization. In the event of an error, targeted queries are then generated so that the movement can ideally be executed.

It is currently possible to give voice instructions to a lightweight robot via a dialog system using the VerbBot prototype for a selection of movements. An extension of the project to a two-arm system and voice-based programming of the system is planned for the future.

Kim Wölfel, Prof. Dr. Dominik Henrich/as

Scientific Society for Assembly, Handling and Industrial Robotics e.V. © MHI e.V.

Briefly explained: The MHI e.V.
The Wissenschaftliche Gesellschaft für Montage, Handhabung und Industrierobotik e.V. (MHI e.V.) is a network of renowned university professors - institute directors and chair holders - from German-speaking countries. The members conduct both fundamental and application-oriented research on a wide range of current topics in the fields of assembly, handling and industrial robotics. Further information on the society, its members and activities: www.wgmhi.de.

Chair of Robotics and Embedded Systems at the University of Bayreuth © University of Bayreuth

Briefly explained: Robotics at the University of Bayreuth
The Chair of Robotics and Embedded Systems at the University of Bayreuth was founded in 2003 by Prof. Dr. Dominik Henrich. It deals with robots as information-processing systems that can detect, change and interact with their environment. One focus of research is the coexistence and cooperation of humans and robots. The aim is to eliminate the strict spatial separation between humans and robots in order to combine their strengths synergistically. Other current focal points are intuitive robot programming and instruction and CAD reconstruction with hand-held depth cameras.
www.ai3.uni-bayreuth.de.