Embedded AI for decentralized decisions

Embedded AI on the rise: market, trends and technology drivers

Embedded AI is increasingly becoming a central driver of modern edge systems. The market is growing rapidly, driven by specialized hardware components such as neural processing units (NPUs) and heterogeneous architectures that are increasingly being integrated directly into microcontrollers and systems on chips (SoCs). This presents engineers with the challenge of implementing complex models on devices with limited memory and computing resources. Techniques such as quantization, pruning and other methods of model compression are therefore essential. In addition to hardware, powerful libraries and tools are becoming increasingly important in order to make AI reliably usable across different platforms.

A concrete example of this trend is the combination of wake word recognition with object recognition and tracking on Qualcomm Snapdragon platforms. YOLOX-based networks are used for object recognition, with inference outsourced to the Hexagon NPU. The NPU is only activated when necessary if a corresponding wakeword (audio signal) was previously given and this was recognized by a second, more energy-efficient NPU. This approach shows how heterogeneous architectures that combine a low-power NPU with Hexagon DSP enable real-time image processing tasks while maintaining energy efficiency.

Designing embedded AI efficiently: Shift Left for smart AI

As the complexity of embedded AI increases, so does the need for clearly structured workflow processes. Model-based design provides a framework for this: Instead of writing low-level code, engineers model their algorithms visually in Simulink. Requirements, models and test artifacts are brought together in a uniform digital thread, which supports collaboration and traceability throughout the entire product life cycle.

A key advantage of this approach is early validation. With the help of hardware-in-the-loop (HIL) and processor-in-the-loop (PIL) tests, potential problems can be identified at an early stage and development can be accelerated, in line with the "shift left" principle. How this approach works can be explained using an example from the automotive industry. In AI-based trajectory planning and control on Infineon AURIX TC4x, neural network controllers are designed and validated using model-based design before deployment. By using a parallel processing unit (PPU) on the hardware, this solution achieves 50 percent higher accuracy and 5 percent energy savings compared to traditional approaches - an example of how structured workflows and hardware-aware optimization deliver tangible benefits.

From Matlab to microcontroller: embedded AI live

MathWorks provides an integrated environment for the development and deployment of AI on embedded systems. Engineers can design and train their models in Matlab or import pre-trained models from frameworks such as 'PyTorch', 'TensorFlow' and 'ONNX'. With the help of automatic code generation tools such as 'Matlab Coder' and 'GPU Coder', these models can be translated into optimized C, C++, CUDA or HDL code for CPUs, GPUs, FPGAs and MCUs. This closes the gap between high-level design and hardware implementation.

Optimization workflows for quantization, pruning and compression are also integrated into the toolchain. They enable use on devices with limited resources without compromising performance. In addition, verification tools such as Polyspace ensure additional reliability by analyzing the generated code statically and dynamically, detecting errors early on in the development cycle - or proving their absence. These functions are particularly important in safety-critical areas to ensure compliance and robustness.

A practical example of this workflow is temperature prediction for electric motors on TI C2000 hardware. Virtual sensor models are developed and trained in Matlab or Python, compressed for use and validated using processor-in-the-loop tests. This approach replaces physical sensors with software-based estimators, reducing cost and complexity while maintaining or even increasing accuracy.

Embedded AI is also already being used in numerous application areas: in the healthcare sector, for example, it is used for real-time signal processing in ECG analysis on STM32 boards. Deep learning and signal processing algorithms are combined here. Sensor data can be processed in real time with the help of automatic C/C++ code generation. Embedded AI solutions are also used in the field of occupational safety: the detection of personal protective equipment on Raspberry Pi and pose detection on NVIDIA Jetson platforms show how embedded image processing and GPU acceleration enable compact, powerful AI solutions for monitoring and compliance.

These examples are indicative of a broader trend: embedded AI is no longer a niche technology, but is becoming a standard for intelligent systems in various industries. Through structured workflows and the use of integrated toolchains, engineers can accelerate their development processes, ensure their reliability and optimize performance for edge deployments. As generative AI and advanced workflows move ever closer to the edge, efficient methods and optimization with hardware in mind will be key to realizing their full potential.

embedded world, Hall 4, Stand 110

The authors: Christoph Stockhammer, Principal Application Engineer, and Dr. Frank Graeber, Manager Application Engineering, both MathWorks