"Thanks to Fraunhofer IPA, we were able to gain an overview of the AI accelerators available on the market in the Quick Check, examine their performance and gain initial impressions regarding the conversion of standard AI models to the target hardware."
Achim Machura
Berghof Automation GmbH
Contact at the AI Progress Center
Christian Jauch
Machine learning with embedded devices
Initial situation
Customers of German machine manufacturers are increasingly faced with the problem that the expertise of female machine operators is steadily declining. Skilled personnel are becoming scarce, so trainees are increasingly replacing skilled workers. AI, especially ML, can compensate for the machine operators' lack of knowledge/experience and significantly reduce training times.
Examples of this include bakery, woodworking and injection molding machines, plant manufacturers in the logistics sector and many more. In all these machines, a PLC usually takes over the control tasks. The market is very price-sensitive, which is why a powerful PC with graphics cards for AI represents a competitive disadvantage. With its MC/BC-PI controller/IPC generation, Berghof Automation GmbH has developed a Raspberry-PI 4-based industrial controller that is used both as a real-time CODESYS controller and as an open IPC solution. EtherCAT as the standard IO bus and a large number of direct
connectable I/Os enable a wide range of applications.
Solution idea (QC)
The use of AI accelerator chips, which can be connected to existing computers such as the Raspberry Pi via a PCI Express interface, should enable significantly lower latencies and throughput rates for a wide range of AI applications. In the context of industrial PLCs and real-time requirements, these AI accelerator chips have not yet been explicitly evaluated, which is why a detailed investigation with regard to these requirements is essential. For this reason, the Hailo-8 AI chip is examined in more detail as part of the quick check and compared with the CPU-only version and the Google Coral Edge TPU chip.
Implementation (QC)
For the analysis of the Hailo-8 AI chip, a benchmark software suite was developed that analyzes AI chips with regard to relevant metrics. These metrics include the latency and throughput of AI inference as well as the CPU load during inference. In addition, several different model architectures are used for the tests. The results of the evaluation show that the potential of AI accelerators compared to pure CPU execution is enormous, especially in terms of efficiency. However, compatibility and user-friendliness vary from AI chip to AI chip.
Solution idea (EP)
The ability to run multiple AI models simultaneously is essential for the use of complex image processing pipelines. Therefore, the capacities of the Hailo 8 chip in combination with the Raspberry Pi 4 were to be analyzed using the example of multi-level activity recognition. In particular, it should be investigated whether the execution latency of the activity recognition is sufficient for a real application such as activity or gesture recognition. To do this, several AI models must be implemented and optimized for the Hailo 8 chip.
Implementation (EP)
An activity recognition pipeline was used to evaluate the Hailo-8 AI chip in greater depth. This consists of an object detector for recognizing people, a pose estimator for estimating skeletal movements and a final activity classifier for determining the actual activity performed. The models were developed and trained in PyTorch. They were then exported to the ONNX intermediate format and finally translated and quantized by the Hailo converter. By using the Hailo-8 chip, it was possible to perform computationally intensive activity recognition with sufficient latency.
Benefit
AI solutions offer a high potential for innovation and cost savings for mechanical and plant engineering, but they cannot be used efficiently on PLCs without specialized hardware. Currently, AI applications on PLCs are not yet widespread. Examples of possible AI solutions that could be made possible by PLCs with special AI hardware:
- Object detection for position detection, room monitoring, etc.
- People and get recognition
- Energy savings through efficient execution of AI models
- Reduce machine downtime through anomaly detection/
- Predictive maintenance to predict failures
- Self-optimizing temperature controls
- Pre-processing of machine data to save bandwidth to higher-level systems
- User support with expert knowledge to compensate for the shortage of skilled workers and shorten training times
- Collective learning (machine A benefits from machine B)