Vehicle part segmentation for improved damage detection

Quick Check

Initial situation

During the visual inspection of vehicles, camera images of the vehicles are analyzed by an AI and defects such as scratches, dents or rim damage are detected. However, as the images of the vehicles are not taken under controlled conditions but outdoors, the surroundings are reflected on the paintwork and in the windows. These reflections can be mistaken for scratches by the AI. False positives occur during damage detection. The damaged component is also an important piece of information when detecting damage. As the repair costs depend on the damaged vehicle part, the repair costs can be estimated more accurately using location information.

Solution idea

AI-based segmentation of a vehicle into individual vehicle parts (e.g. door, hood, bumper, rims, tires, mirrors) can increase the reliability of the already developed AI surface inspection and vehicle documentation. This is because certain areas of the vehicle have typical defects (e.g. rim damage) that only occur there. Other areas can remain unconsidered during the inspection, thereby avoiding misclassifications (e.g. reflections of the surroundings in the vehicle windows or typical soiling on the mudguards in winter).

KI-Fortschrittszentrum_218_QC_Fahrzeugteilesegementierung_Abb1 — Figure 1: Detected vehicle damage and reflections, Two Tronic

Benefit

Vehicle part segmentation allows the individual defects detected by the AI to be assigned to the vehicle parts. This information is now used to recognize incorrect detections of damage (false positives) by defining specific filters for each vehicle part. In this way, prior knowledge can be incorporated into the damage detection process, thereby reducing the system's susceptibility to errors. Vehicle part segmentation can also be used to determine the damaged component and enter it in a damage database. As the repair costs depend on the damaged component, this information is important for assessing the damage.

Implementation of the AI application

A representative data set of 300 vehicle images was created for the vehicle part segmentation. Based on this data, an AI was trained to segment individual vehicle parts. The vehicle was divided into ten classes, such as "hood", "rim", "rear light" or "windows". The individual pixels of the vehicle images are now assigned to one of the ten classes or the background by the AI. This information is combined with the detected damage in the further course of the program, so that the accuracy of the damage detection is improved and the damaged component can be stored in a database.