Machine learning in feeding technology

Modern production systems are exposed to a variety of challenges due to high cost pressure combined with increasing numbers of variants and increased planning uncertainty. In automated assembly, these challenges can be met by using flexible equipment. Feeding technology, as a crucial component of automated assembly systems, must also meet these challenges. However, conventional feeding systems such as vibratory bowl feeders have a low level of flexibility due to the nature of the system. One promising approach being researched at the Institute of Assembly Technology (match) at Leibniz Universität Hannover is aerodynamic feeding technology. Here, the components are brought into the correct orientation for the subsequent handling and assembly steps using targeted pulses of compressed air. Compared to the use of mechanical baffles, this process offers greater flexibility and reliability in operation.

In order to enable the targeted manipulation of the component orientation by the compressed air pulses, the orientation or pose of the component must first be determined. In this context, match is researching a component-independent method for classifying possible component poses using artificial neural networks (ANN). Data sets with several hundred images of each component orientation are required to train the neural networks. To create the training data sets using the conventional procedure, the components in the feed system would therefore have to be photographed several thousand times in total. In order to significantly reduce the effort and costs involved in creating the training data sets and avoid set-up times altogether, the KNN is trained using artificially generated data sets.

Training with synthetic data

CAD components in standard STL format serve as the basis for creating the synthetic data. Based on this data, the simulation-based determination of the possible orientations that the component can assume in the feeding system is carried out first. For this purpose, virtual drop tests are carried out in a physics simulation and the resulting component poses are automatically evaluated and clustered. In the real feeding system, the separated components are guided sideways on the conveyor belt so that they can only assume a limited number of discrete poses when passing the high-speed camera, which must be classified by the image processing system.

The identified component poses are then transferred to a framework in the open source software Blender. In Blender, the components are placed in a model of the feeding system and a data set with several hundred images is automatically rendered for each component pose. The component position, exposure and camera position are varied slightly in order to increase the robustness of the KNN trained with the data set. The ANN is then trained using only the synthetic data set. By varying these parameters during rendering, overfitting of the network is avoided and the network can be used directly for inference, i.e. the classification of real component poses in productive operation.

High accuracy with a simple framework

The machine learning framework does not require any proprietary software and can be easily transferred to other applications. The parameters for generating the synthetic data and the neural networks used can be adapted or exchanged without any programming effort. To validate the framework, investigations were carried out into the classification quality of various network architectures. The investigated networks were trained exclusively with synthetic data sets and then applied to real data (photos of the component poses). For the component shown in Figure 1, the so-called SqueezeNet, a CNN pre-trained for image processing, was able to correctly classify the different poses with an accuracy of over 99%. An alternative architecture, the MobileNetV3, achieved an accuracy of over 98%. Of a total of ten components examined, half achieved an accuracy of over 90% and seven achieved an accuracy of over 80%. This shows that it is possible to classify real components using ANNs trained with synthetic data. At the same time, there is still room for improvement, especially for components with less pronounced features, the classification accuracy is still below 80%. Further investigations are aimed at closing this gap and thus enabling universal use of the machine learning framework presented.