In the search for high-quality solutions, companies are also finding what they are looking for in Germany. Experts from the Fraunhofer-Gesellschaft are developing voice assistants for business and industry that place the highest priority on data protection as well as performance and security.

When Gerrit Holzbach critically examines the surface of a hood, he discovers a defect and points to the spot. "There's a matting," he notes. "Okay, matting," confirms not a human colleague, but a multimodal assistant, or "MuDA" for short. The system marks the fault by projecting a dot of light onto it and immediately starts the repair when Gerrit Holzbach says "Repair this fault". A robot arm approaches the marked spot with millimeter precision and treats it with its polishing attachment. "Repair of the matting defect has been completed," reports MuDA, already waiting for the next job.

Dialog systems such as the multimodal dialog assistant demonstrate this: New forms of interaction with technical devices via voice have great potential to relieve employees in companies, speed up processes and increase the quality of work. AI-based voice solutions are already being used in various industries - as one of Europe's leading scientific institutes in the field of artificial intelligence, the Fraunhofer Institute for Intelligent Analysis and Information Systems IAIS is developing such voice technologies for different requirements and areas.

"We work together with various companies, determine their respective needs and then develop customized solutions," says Oliver Walter, an expert in speech technology at Fraunhofer IAIS. For example, software for speech and speaker recognition from Fraunhofer IAIS has already been in use in the media industry for several years: with an audio mining system specially developed for ARD, employees can search for terms or people within videos or quickly find relevant content such as articles or files in archives.

Other voice assistance or dialog systems, such as the multimodal assistant MuDA, combine several speech technologies. One component is speech recognition. This is followed by speech understanding, also known as Natural Language Understanding (NLU). "As the name suggests, NLU systems make it possible to understand and interpret texts or spoken language with the help of AI. This saves companies time and makes work processes more efficient," explains Oliver Walter. "However, AI systems must also be able to interpret the queries correctly and respond with specific specialist knowledge, especially in industry." So-called knowledge graphs are used here, which encode the relationship between words and their meaning. This allows complex knowledge to be linked in a network and interpreted by machines. Finally, speech synthesis converts text into spoken language. "Many people are already familiar with this component from smart home assistants, which not only receive our commands and questions, but also respond to them," says the Fraunhofer scientist. This "question-answering" technology also has enormous potential in working life, for example when it comes to retrieving specific knowledge from large databases in a short space of time.

Voice and gesture control for natural interaction

The MuDA assistant goes one step further and combines speech recognition with gesture control. The development from the Fraunhofer CCIT Machine Learning Research Center can be used in quality assurance in industry, especially for identifying and marking defects on surfaces. "Quality assurance is an important part of production processes, but at the same time it is often complex, time-consuming and cost-intensive," says Walter, who heads the MuDA project at Fraunhofer IAIS. In order to meet these complex requirements, a robotic system from the Fraunhofer Institute of Optronics, System Technologies and Image Exploitation IOSB was further developed into a multimodal dialog assistant in cooperation with the Fraunhofer IAIS. It enables automated error documentation and handling, and supports humans from marking to repair.