Adapting a trained model for new sensor sources

Quick Check

Initial situation

Neural networks that are to be used for autonomous driving of motor vehicles are often trained by manual driving. This involves a measuring system recording data from sensors, which is then used to train the neural networks. The placement and calibration of the sensors plays an important role here: when using the generated internal models of the neural networks, for example, the same sensors must be used at the same installation point and with the same external parameters (direction, location) that were used during training.

Solution idea

The use of already trained models and their adaptation to other contexts could offer advantages in the technology development process. For this reason, it will be investigated whether it is possible to extend the use of the Comma.ai open source pilot system by using sensors captured by heavy vehicles without major losses in terms of performance and accuracy.

Figure 1: Concept of the AI application, Knorr-Bremse GmbH, Fraunhofer IPA

Benefit

The application of these AI models to trucks would make a significant contribution to saving costs and effort in the development of autonomously driving trucks and thus enable the development and deployment of such systems in a shorter time. Among other things, this reduces the need to drive further distances with trucks in order to generate and train their own models for use in trucks. This then contributes to sustainability, among other things, because energy consumption and CO2 generation are reduced.

Implementation of the AI application

In this quick check, the first concept was verified using the open source pilot system from Comma.ai as a test system. In this case, the data originally captured on the car was replaced by images from the cameras and sensors supplied by Knorr-Bremse, which were captured from trucks. This data was adapted to the specifications of the original neural network, which remained unchanged in this first concept. The results are that the neural network, although trained for a different type of vehicle, provides a correlated output in terms of accelerator pedal pressure, brake pressure and steering angle with respect to the input provided.