Automating defect recognition

Do we need to know what is wrong to recognise what is right?

Even now, production line quality control continues to be a manual process that involves the extensive use of resources, both economic and non-economic, and that may be prone to errors. Is it possible to automate this process through the use of images and video? Certainly, but developing a machine learning model usually involves a large number of images that are needed to train the model. This may not be a problem if, for example, you have a factory producing millions of items per day and you have the option of installing a camera and capturing a large number of images of the product. The difficult issue is that such algorithms usually require images both of defective products and non-defective products. While correct products will be similar to one another, most of the defective products will be unique and unpredictable in terms of their differences, making it extremely difficult to have a set of images that represent them completely. The system created allows for the detection of anomalies without having to use images of defective products during the training process. This makes it possible to set up a quality assurance system that learns what products should look like and how to detect any deviations from the norm. In other words, we need to know what is right rather than recognising every incorrect form of the product.

The solution you have dreamed of

The solution that has been developed consists of two components. The first involves training the machine learning model using only correct examples of the product. To achieve this objective, a deep learning algorithm was used known as an “autoencoder”. The second aspect relates to how images might be captured from a production line at high speed and prepared for the machine learning algorithm. To explain how the solution works, a version of which was built and successfully tested in an industrial setting, imagine that we need to identify the defects on a production line used for very small confectionery products, such as mints.

The magic of the autoencoder

The heart of the solution is a type of neural network model termed an autoencoder, trained on a large number of images. The special feature of the method is that during the neural network training phase, only the images of non-defective products are used, with no defect categories involved. With this approach, the final model can therefore detect any type of deviation from the normal appearance of the product: both already identified defects and new defects. An autoencoder, depicted schematically, is a special type of artificial neural network capable of codifying objects in an unsupervised learning context. When an item of data is supplied to the autoencoder without specifying its significance, the autoencoder autonomously learns which characteristics are typical of that data item and which are not. An autoencoder consists of an encoder and a decoder. The encoder receives the input data and transforms it into an internal representation. The decoder accepts this internal representation and builds the output from it. The layer containing the internal representation is generally smaller than the input and output layers and is termed the bottleneck layer.

Autoencoders are trained to generate output data that is as similar as possible to the input data. This is similar to a “copy and paste” operation with a small intermediate hidden stage. The secret of the autoencoder lies in the layers that contain the internal representation. These layers form a bottleneck for the information contained in the training data. Since the bottleneck layers are generally much smaller than the input layers, the autoencoder code is forced to gather only the essential features of the data. Such features can then be used to identify anomalies: when an anomaly occurs, the automated encoder will be unable to reconstruct it efficiently since, during the training process, no structure was developed within the neural network that could represent the anomaly itself. If the autoencoder’s output data is then compared with its input data and there is a large difference, it can be assumed that an anomaly exists. The autoencoder automatically “subdivides” the defects shown in the image, such that the various data points indicate the type of defect involved. These deviations might provide more detailed information on the defect type (such as “break” versus “indentation”).

How do we get from the algorithm to real-life defect detection?

The second component of the solution relates to the way in which the autoencoder algorithm is incorporated into the mints production line. The identified solution involves recording the images of the confectionery during the production process. In view of the fact that the images recorded may contain more than one mint at a time, they need to be cropped so that only a single product can be seen. The availability of images showing the individual product thereby enables the autoencoder to train itself.