Contact at the AI Innovation Center
Andreas Frommknecht
Perception for automated handling of dimensionally stable components (AutoLab)
Overview
Fraunhofer IPA aims to establish a project with creative and proactive industrial partners from Baden-Württemberg who share common interests and wish to exchange experiences.
In the project "Perception for the automated handling of dimensionally stable components (AutoLab)", Fraunhofer IPA shares its expertise, particularly in the field of artificial intelligence.
intelligence (AI), with the partners and provides insights into their own areas of application. The industrial partners can thus transfer the benefits of using AI to their own applications and benefit from the project results. The project is scheduled to start in January 2022 and run for 12 to 18 months.
A wire harness is the most expensive part in the assembly of electromechanical systems. Almost all assembly processes with complex and varied components are carried out manually. It takes a lot of time and effort to understand customer specifications and design them according to requirements. This tedious task is a bottleneck in the production of electromechanical systems. Perception for automated handling of dimensionally stable components (AutoLab) enables the assembly process to handle even complicated cases correctly. The AutoLab project consists of three main processes: Perception, Handling and Inspection. The different components are automatically classified by the perception system using machine learning. The handling system then performs and optimizes the assembly process based on the identified requirements. Finally, the results of the assembly are monitored in the inspection phase to check that they have been carried out correctly.
Perception
In the conception phase, customer specifications are analyzed using artificial intelligence. A 2D drawing plan designed by the customer describes their requirements, e.g. the arrangement and type of assembly parts and at which angles the objects are to be installed. The different types of information in the 2D drawing are extracted by object recognition, as shown in Figure 2, and optical character recognition (OCR). The object recognition method recognizes instances of objects of a certain class and perceives the position of the objects. In addition, the size or angle of the object is captured by the OCR technology. All captured data is calculated to map the relevant information in the right place. This calculated data is forwarded to the next step so that the robot can perceive the requirements.
Handling
The extracted customer requirements for wire harness assembly are used as input information for the handling module. In this module, the product requirements as well as the positions and tolerances of additional components are used to model the final product. This is transferred to a physics simulation and provided with a robot cell and a robot. This simulation environment forms the basis for the machine learning algorithm. Parts handling and the subsequent assembly of the final product are controlled using a hybrid learning approach. This consists of a classic robot control algorithm and current "stateoftheart" machine learning algorithms. The assembly process is learned completely offline, resulting in cost-efficient training, continuous production without interruptions and early improvement loops. Several methods are used to transfer the learned control algorithm into reality, including domain adaptation and domain randomization. The combination of these methods allows high-frequency
Production adaptations due to the flexibility of the approach, the simultaneous learning of multiple requirements and the simplicity of adapting the automated pipeline.
Inspection
During this phase, the finished product is automatically inspected using an industrial camera and suitable lighting. The results of the assemblies are recorded by the camera and analyzed using a machine learning process known as a convolutional neural network (CNN).
In a quick check project with an industrial partner, the possibilities for detecting faults based on customer requirements with high prediction accuracy were demonstrated. The characteristics such as alignment and positioning of components were defined by the industry partner. With the defined requirements, the trained model achieves 100% accuracy in the test set. The reliable results can also be used to optimize the parameters of the handling module.
