Evaluation of the relevance of image data in the training of neural networks

Quick Check

Initial situation

When training AI models and neural networks in particular, large amounts of data are required. The storage requirements and training time for images in particular are therefore enormous. Current research shows that not all training images are equally valuable. For example, low-quality data has no or even a negative impact on AI models and can be removed without any loss of accuracy in order to reduce costs. IDS offers its customers industrial cameras and a platform in which customers can train AI models according to their own requirements. The aim of this project is to find a method that can identify low-quality data and allow users to remove it from the deep learning process at an early stage, ideally during image acquisition on the intelligent camera.

Solution idea

A number of publications have addressed the topic of data value in recent years. A Google paper from 2020 stood out in particular. Reinforcement learning is used to train a neural network to evaluate the quality of images and their annotations. In the experiments described, the results were convincing. However, in order to use the method on the camera, it has to work without annotations. It therefore remains to be clarified whether the method works on image data and whether it is stable even without annotations.

Figure 1: Evaluation of the quality of training images and subsequent weighted training, Danilo Brajovic, Fraunhofer IPA

Benefit

The performance of an AI model can be increased enormously by removing corrupted images. By evaluating the quality of an image at an early stage, the upload of the image to the server and the manual relabeling of the image can be avoided. The manual labeling of images is particularly time-consuming and cost-intensive. This would make it easier and quicker to use many AI processes in practice.

Implementation of the AI application

A ready-made evaluation model could run either on the camera or on the server. On the camera, it would prevent low-quality images from being uploaded in the first place. If the application is too computationally intensive, it could run on the server to avoid manual relabeling or to identify images that have a negative impact on training performance.