In a research project, A&T Manufacturing, SHL and the Institute of Production Engineering and Machine Tools (IFW) are developing an image-based quality control system with the use of artificial intelligence (AI).

High potential through AI-based evaluation of ground surfaces

In addition to the technical function of surface grinding, some applications also require a decorative effect of the grinding pattern. Typical applications include fittings in automobiles and visible profiles in interior fittings. Reliable quality control is essential in order to meet the high demands on the uniformity of the surface finish. Nowadays, this assessment is often carried out without the aid of a manual visual inspection by production personnel. However, this is accompanied by the deficit of cost-intensive and monotonous inspection processes. This deficit is countered with an automated quality inspection. A&T Manufacturing, SHL and IFW Hannover are therefore working together to develop an image data-based AI system that enables automatic quality control of decorative micrographs without contact and within a few seconds. The image data is captured by an industrial camera. The image data is processed by an AI algorithm, which evaluates the surface texture.

The image processing process shown in Figure 1 is used to categorize the four classes. The input of the system is a robot-guided industrial camera with a CMOS surface sensor, which takes individual images of the surface step by step. During image acquisition, a constant illumination is realized by a surface light close to the component in order to increase the robustness of the system against interfering light from the environment. A Convolutional Neural Network (CNN) is used for image processing, which extracts relevant features from the gray values of the image data. The structure of these networks has proven to be particularly effective in image processing applications. A CNN is used to classify the individual images into the four classes, which evaluates each image individually and saves a class classification as an output. This classification is made possible by a previous training process with image data of the texture classes. As the components are larger than the image capture area, it is necessary to capture the component segment by segment. For the quality output of the system, the recording position of the individual images is therefore assigned to the individual component segments. The quality class is therefore available for each recorded component segment.

To evaluate the trained CNN, a confusion matrix (Figure 1 below) was created with 100 previously unknown test images. The matrix graphically shows which classes were predicted by the CNN for the test images and which true classes are present. All 25 test images of the classes "OK", "not OK" and "raw part" were correctly predicted. The CNN therefore makes extremely reliable predictions for these classes and is suitable for differentiating between these three classes.

Of 25 test images in the "no component" class, however, 13 test images were incorrectly assigned. The system is therefore unable to recognize when there is no component in the image area based on the image features alone. The reason for this is that the image acquisition in this case is not focused and the lighting is irregular, which favors an incorrect prediction. In order to increase the reliability of the system in this case, a distance sensor will be integrated into the robot end effector in future. This is used to check whether a component is actually present in the camera's recording area.

Summary and outlook

A robot-guided camera system with image evaluation is being developed for the automated quality control of decorative micrographs. The evaluation algorithm classifies the texture of the surface using trained CNNs. The results show that the classes "OK", "not OK" and "blank" are reliably predicted. However, an additional distance sensor is required to detect the "no component" class. Up to now, the developed system has not taken into account the fact that isolated surface scratches can lead to the entire component being rejected. Such scratch detection is currently being implemented with a CNN for the detection of objects. The entire image processing system will then be integrated into an industrial process chain for the production of decorative surfaces.

Prof. Dr.-Ing. Berend Denkena, Dr.-Ing. Heinrich Klemme, M.Sc. Jan Geggier, IFW Hannover

The research project "Adaptive robotic grinding cell for the production of decorative grinding on complex-shaped aluminum profiles" is funded by the Federal Ministry of Economics and Climate Protection (BMWK) on the basis of a resolution of the German Bundestag as part of the Central Innovation Program for SMEs (ZIM) and is supervised by the German Federation of Industrial Research Associations "Otto von Guericke" (AiF). A&T Manufacturing, SHL and the IFW are grateful for the financial support in this project.