Application of machine vision for the detection of powder bed defects in additive manufacturing processes

Additive manufacturing makes it possible to produce a physical object directly from a computer-aided design (CAD) model. This speeds up the production process, especially for small batches, and allows for more freedom in design. The most common process group for metal parts is powderbed fusion, which includes selective laser melting (SLM). In SLM powder, the powder layer is spread repeatedly on the building platform and then locally melted with the laser source. The process is then repeated until the finished part is produced [1]. The quality of the item produced is affected by several factors, including the exemplary laser power, scanning speed, and the direction of the building [2].

One of these factors is the quality of the powder layer, specifically uniformity and powder packing density. The variation in powder layer thickness may result in under- or over-melting due to difference in needed volumetric energy density [3].

Powder packing density, in turn, affects the laser absorptivity and final density of the parts [4]. Common problems with powder layer uniformity include recoater streaking, recoater hopping, incomplete spreading, debris, etc. [5]. Problems intensify in the case of powders with low flowability caused by the existence of irregular or elongated particles [6] or having magnetic properties [7]. Such materials include iron-based metallic glasses that are often characterized by soft magnetic properties [8] and are extensively applied in the SLM process [9]. The control of powder spreading allows for an early response in the event of a defect occurring.

Various thermal, optical, or acoustic sensors are already widely adopted in SLM. They enable a detailed characterization of the build process and prediction of the properties of printed parts. The development of sensing systems is headed in the direction of machine learning algorithms that detect anomalies in closed-loop control [10]. Existing research also addresses the system for monitoring the quality of powder layers. Craeghs et al. [11] developed a visual inspection system to detect the wear and local damage to the coated blade, which causes deep local scratches in the powder bed. It is desirable to identify such defects before melting because after that point, the quality of the part will be irreversibly affected, for example through increased roughness.

Yin et al. [12] focused on detecting recoater hopping. This defect, which manifests in the form of vertical lines, is caused by the recoater hitting the part or trembling [5]. The detection is based on the camera and the local binary pattern (LBP) recognition algorithm. The highest recorded recognition rate was 98% when using sufficiently small partitions [12]. Scime et al. [5] trained the algorithm to detect defects and classify them into separate categories on the commercial EOS LPBF machine. Later, they improved the algorithm [13] in terms of defect localization accuracy.

In this work, the goal of comprehensive powder bed monitoring is presented, as well as an algorithm that implements contemporary machine learning and computer vision techniques to detect and classify the enumerated anomalies using only limited hardware. This work fits into the current trend of applying image-processing algorithms to monitor the quality of the powder layer. For this purpose, MVTech HALCON software was used, which is a comprehensive solution for machine vision. The trained support vector machine (SVM) algorithm convolutional neural networks (CNN)) make it possible to detect powder bed defects. This is especially useful in the case of problematic powders and in the development of new feeding strategies for them.

Artificial Intelligence (AI) and Machine Learning (ML) algorithms are increasingly used to analyze various technological processes. Trends in modern production techniques use Additive Manufacturing (including 3D printing) for the unit production of details with a specific chemical composition and shape. Due to the relatively high costs of AM, quality control at all stages of 3D printing is highly desirable. The scientific aim of the conducted research was to determine the possibility and usefulness of using machine learning algorithms to assess the quality of the metal powder distribution process in the working field in metal 3D printing. Due to the preliminary nature of the conducted work, the authors focused on generating correct (OK) and incorrect (NOK) cases of powder distribution, without determining the impact of input parameters such as scraper movement speed, or its degree of wear or positioning accuracy. In order to obtain NOK cases, disturbances were consciously and deliberately introduced into the process: inaccurately and worn positioned scraper, too little powder fed by the feeder, and vibration of working field.

The testing of the vision system might be realized in commercial SLM machines [14] or with our own developed powder feeder prototype [15]. In this work, our own powder feeder was constructed. This allows for greater control and adaptability in implementing the vision system, as well as allowing for modification of the system to improve the quality of the obtained powder layers. Moreover, the various powder feeder concepts planned in the printer under construction might be monitored. The algorithm for detecting defects in the powder layer was tested first with a classical powder feeder and distribution system based on two platforms. One platform served as the working platform (the socalled powder bed), and the other platform was the so-called dispensing platform, which served as the powder feeder. The dispensing platform was sequentially moved up to feed a specified volume of powder, while the powder bed was lowered by a preset layer thickness so that another layer of powder could be applied for sintering. Due to its simplicity and low requirements for the amount of powder, this technical solution is very often used in commercial 3D metal printers.

The principle of the feeder system used in the application shown is as follows:

Activation of both platforms (moving the dispensing one up and the working one down) simultaneously.

The advantage of such a solution is that it takes up minimal space outside the construction surface.

The presence of a control system coupled to the drives makes it possible to precisely adjust the amount of powder fed, the speed of the scraper, and the thickness of the sintered layer (by setting the distance at which the working platform is to lower).

The view of the working platform with the scraper in the form of a blade is shown in Figure 1.

Fig. 1.

The Balluff BVS002C smart camera was used for image acquisition and image preprocessing. Using wired LAN communication and the Ethernet protocol, it is possible to communicate with the camera and program it using a web browser. The camera’s built-in graphics processor and software enable all image operations to be carried out directly on the camera, with the connected PC serving only as a programming terminal. Once the programming stage is completed, the camera becomes an autonomous device, and it is possible to operate it without connecting a PC.

An unquestionable advantage of the Balluff BVS002C camera is its support for the PROFINET protocol, which makes it possible to send and receive data (control signals, data acquisition results) to control or execution devices (e.g., PLC, robot, other devices supporting PROFINET). In addition, the camera supports the I/O-link automation standard, which allows for direct parameterization of executive devices (e.g. RGB lamps visualizing the obtained results) and direct transmission of selected measured values to the PLC.

The basic parameters of the camera used for image acquisition during the tests are:

image resolution: 1280 x 1024 px,

Figure 2 shows a Balluff BVS002C camera installed above the work field and integrated with a Balluff BAE000K ring LED illuminator. The lighting strategy is one of the important factors affecting the possibility of detecting defects in the powder bed [16].

Fig. 2.

Dozens of tests were conducted for various parameter settings from the controller and direct modifications to the feeder on the fly. For each test, 6-7 passes of the system were made, camera images were captured, and potential irregularities in the powder fed were analyzed. The variable parameters set by the controller, which controls the drive of the powder feeding system, are:

roller speed,

The program sequence for testing the powder distribution system was as follows.

Engagement of both platforms’ drives simultaneously, advancing the dispensing platform and lowering the working platform.

Different combinations of the process parameters were used to obtain different quality parameters of the distributed powder.

As soon as the metal powder had been distributed by the scraper system control unit (PLC), the Balluff camera model BVS002C was triggered to begin image acquisition. Dedicated BVS Cockpit software allowed the camera to capture images of the layers, and also, after implementing an image recognition algorithm, to detect and measure the unevenness of the layer of distributed powder.

First, the analysis area was restricted to the region where the powder is analyzed (Figure 3A). The image was then processed by thresholding (binarization), in which the pixels are subjected to comparative analysis. Pixels brighter than the set threshold become white, and the remaining pixels become black, respectively taking the values of 1 (white) and 0 (black). In the next stages of image processing, a segmentation algorithm was implemented (Figure 3B). Regions of the image with the same characteristics - in this case, with a shared histogram-defined brightness – were extracted.

Fig. 3.

Using the Balluff BVS002C vision system integrated with the powder feeding system, observation of the working field was carried out. Views of different distributions of powders using different processing variables are shown in Figure 4.

Fig. 4.

Figure 5 presents examples of how segmentation was used to extract features corresponding to irregularities in the powder layer. The segmentation parameters were chosen empirically to expose the irregularities created from the scraper’s movement.

Fig. 5.

The algorithm for processing the image captured by the camera consisted of several consecutive steps (shown in Figure 6). (1) First, the correct image exposure conditions were set – i.e., the shutter speed. (2) Then the background elimination algorithm was implemented, which consisted of searching for arcs on the border of the working platform. (3) The arcs were closed with a circle defined by the algorithm. In this way, the analysis area was automatically narrowed down to the inside of the circle with a diameter corresponding to the diameter of the working area. (4) In the next step, a preliminary segmentation of the image was performed, extracting morphologically homogeneous areas in terms of brightness. This allowed for detection of potential irregularities in the distribution of the powder through the feeder. (5) In the final step, criteria were applied based on subjective evaluation to isolate the most critical areas in terms of quality.

Fig. 6.

Figure 7 shows an example view of the input and output images for a case in which the powder was distributed incorrectly. It should be noted that the algorithm, when run automatically (on the basis of predefined criteria), allows for qualifying the process of powder layer distribution as OK (correct) or NOK (incorrect).

Fig. 7.

Figure 8 shows an example view of the input and output image for a case in which the powder was distributed correctly and classified as OK.

Fig. 8.

The presented method for detecting irregularities in the powder layer is based solely on classification using conditional instructions, the inputs of which are the regions recognized in the segmentation stage. To increase the reliability of the algorithm, the machine learning algorithm SVM (Support Vector Machines) method was used. This method is based on the concept of decision space, which is divided by building boundaries separating objects belonging to different classes (e.g. good/bad).

A detailed block diagram of the algorithm, taking into account the operations of acquisition, image processing, and classification, is shown in Figure 9.

Fig. 9.

The algorithm shown in the block diagram uses functions and procedures available in the HALCON library. The algorithm consists of the following proposed steps, outlining the four main stages.

Acquisition of data for analysis and classification.

To start the process of creating a classification model, it is necessary to define the input data (images of the powder layer) and sort them into three groups:

images with a properly distributed powder layer,

Segmentation is the process of dividing an image into areas so that characteristic parts of the it can be extracted from the whole. In this case, at the beginning, the segmentation process consists of dividing the image into 5 identical channels in which a different filter was applied to each of them. The selected filters performed operations such as edge detection, extracting ripples, wrinkles and closed areas, and detecting differences in contrast, among others. These operations were realized by a twodimensional convolution according to a generalized expression [17]: y[m,n]=x[ m,n ]∗h[ m,n ]=∑j=−∞∞∑i=−∞∞x[ i,j ]⋅h[m−i,n−j] \[\begin{matrix} y[m,n] & =x[m,n]*h[m,n] \\ {} & =\underset{j=-\infty }{\overset{\infty }{\mathop \sum }}\,\underset{i=-\infty }{\overset{\infty }{\mathop \sum }}\,x[i,j]\cdot h[m-i,n-j] \\ \end{matrix}\] where:

x[m, n] = input image,

The filtration stage is followed by the compositing of all five channels, from which one image is created.

The images prepared in this way are added to a classifier implemented by a ready-made function from the HALCON library. The activity is repeated in a loop for all input images determined to be valid. The most well-known methods for creating a classifier include:

SVM – Support Vector Machines,

For the presented application, an SVM classifier has been used [18], characterized by the possibility of so-called “novelty detection”, which allows for a search in the image for places that differ from those of the classifier with trained samples. In this way, it is possible to recognize regions of images that deviate from the accepted norm. The SVM method is a part of traditional machine learning algorithms, which prioritize predictive performance over interpretability. It can capture complex, nonlinear relationships between predictors and the response variable. However, the raw data has be transformed into a suitable internal representation that captures relevant patterns and information for the models to learn from.

In order to increase the reliability of the algorithm, deep learning was applied using the “MVTec Deep Learning Tool.” To achieve high prediction accuracy, machine learning models often require a large amount of training data, but this can decrease their interpretability. Unlike conventional machine learning, deep learning models can automatically extract features from raw data, but this adds complexity and makes the results more difficult to interpret [19]. The “MVTec Deep Learning Tool” offers overtrained neural network (CNN) models. The two-stage convolution neural network (TS-CNN) model has been already successfully implemented in monitoring defects in real-time during selective laser sintering (SLS) [20]. The CNN-based models allow also for the incorporation of a multi-sensor fusion approach, aggregating the layer-wise images, acoustic emission signals, and photodiode signals for achieving in-situ quality monitoring of the SLM process [21].

In this paper Deep Learning Anomaly Detection, which is considered to be unsupervised learning, was used to investigate the quality of the distributed powder on the 3D printer’s working field. Training of the model was carried out only on images marked as correct (OK). Anomaly detection of areas of images that were significantly different allowed them to be classified as incorrect (NOK) images. The learning was carried out using an NVIDIA GeForce RTX 2060 graphics card. The images were divided into three categories – training, validation, and testing – in order to select optimal hyperparameters during the learning process. To optimize the learning process time-wise, the images that were used after pre-processing involving cutting out the nonpowder area distributed on the table, and the resolution of the images was reduced to 256 x 256 pixels. In addition, data was augmented using changes in reflection, rotation, brightness, and/or contrast.

Figure 10 shows the results of deep learning, along with the plotted error maps that represent the probability of incorrect powder application.

Fig. 10.

The quality of the powder distribution process in the working field during 3D printing process is extremely important and corresponds directly to the quality of structures made with this technology. Therefore, at the prototyping stage, the control of the distributed powder is already extremely important: it allows for the detection of defects and irregularities at an early stage in the production of printed structures. One of the most promising and easy-to-implement methods is visual inspection combined with the use of machine learning algorithms. Research conducted by the authors has proved the high potential of image processing, machine learning and artificial intelligence methods for assessing the quality of powder distribution in the metal 3D printing process. The next stage of this work will be the development and implementation of machine learning algorithms, thanks to which it will be possible to identify specific input process variables resulting in incorrect powder distribution.

As a result of the conducted research, it was found that:

Vision systems, image processing algorithms and machine learning are suitable for assessing the quality of the metallic powder distribution before the laser sintering stage as part of the 3D printing process.