Automatic recognition of density and weave pattern of yarn-dyed fabric

With the development of society, small-batch and multi-item have gradually become the production mode of the textile industry, which means that the textile companies are in desperate need of a rapid, automatic, and accurate method for analyzing the structural parameters and weave patterns of the fabric. In the traditional manual method, these parameters are identified by the inspectors with a textile magnifying glass, which is time-consuming and labor-intensive. Along with this trend, the recognition of fabric parameters, such as density and weave pattern, has become a research hotspot. Many feasible algorithms [1,2,3,4,5,6] based on image analysis have been proposed to recognize the fabric density and pattern. Generally, these methods can be classified into two categories: hand-crafted feature-based methods and deep learning-based methods.

The low-level features, such as the frequency and time domain features of the image, are often used to represent the texture information of the fabric. In the frequency domain, the peaks in the Fourier transform results of the fabric image represent the periodic structure formed by warp and weft yarns. Automatic detection [7,8,9,10] of fabric density can be achieved by locating the peaks in the fabric image and counting the number of located yarns. In the time domain, the pixels in the yarn have higher gray levels, while the pixels in the gap between the yarn reflect fewer lights, showing a lower gray level. In ref. [11], gray projection was proposed to locate the nodes and yarns. Then, pixel-level features, such as geometrical features [9,12,13] and statistical features [14,15], are extracted to classify the nodes. However, these methods pay more attention to white or homochromatic fabrics.

Recently, a significant breakthrough has been achieved on image analysis by moving from the low-level feature-based algorithms to deep learning-based end-to-end framework. Convolutional neural networks (CNNs) are widely applied to solve pattern recognition tasks. In the textile field, many researchers also adopted CNN methods to address some visual tasks, such as fabric image retrieval [16,17,18], weave pattern recognition [1,19,20,21], and defect detection [22,23,24]. For fabric weave pattern recognition, some researchers [20,25] converted the task to a classification task. However, Meng et al. [1] regarded the pattern recognition task as a multi-task problem that needs to locate and classify the nodes. Technologically speaking, fabric density and pattern recognition are carried out by many tasks, such as regression, classification, object detection, and color measurement. The complexity of this task makes it difficult to train and learn from the end-to-end CNN model. Moreover, the CNN methods require a lot of labeled data to drive learning, but it is difficult to obtain many ground-truth. This is the main reason why there are still few studies based on deep learning to address the weave pattern problem.

With the high demands for the appearance of clothes, yarn-dyed fabrics play a more important role in the textile market. Due to the mutual influence of various colors in the image of yarn-dyed fabric, it is difficult to achieve good performance by the current methods. Pan et al. [26] proposed a method for automatic detection of structural parameters of yarn-dyed fabric based on Hough transform and fuzzy C-means (FCM) algorithm. Xin et al. [27] used the active grid model to recognize the color pattern of yarn-dyed fabric. However, if yarn-dyed fabric has weft skew, these methods may not work well. Based on the work of previous researchers, in this article, an efficient method is proposed to recognize the structural parameters of yarn-dyed fabric, including density, weave pattern, and colored yarn arrangement. The proposed method can help companies analyze fabric parameters more quickly, resulting in efficient production. The following section introduces the proposed method in detail.

Woven fabrics are produced by interlacing two perpendicular sets of yarns: vertically passing warps and horizontally wefts. The cross states of warps and wefts are called interlacing nodes, which are classified into two types as follows: the warp interlacing node denotes a node with a warp residing on the top of a weft, and the weft interlacing node refers to a node with a weft passing above a warp. Generally, the weave pattern consists of the recurrence of the basic weave repeat. Yarn-dyed fabric, as a common type of woven fabric, is woven by two or more different colors of yarn.

The stability of the acquisition environment, including lighting conditions, scale, resolution, etc., is essential for fabric pattern recognition. For this requirement, the scanner is a good choice, which is not only cheap but also convenient. In this study, Canon 9000F Mark II Scanner is used to capture the yarn-dyed fabric image in the RGB (red, green, blue) model. The light source of this scanner is a white light-emitting diode, which can guarantee a stable capture environment. When obtaining fabric images, the resolution is an important parameter. A too high resolution will increase the amount of calculation in the recognition process, thereby affecting the recognition speed; whereas a too low resolution will cause inaccurate recognition. According to the warp density (weft density) of the fabric of 800 pieces/10 cm and the maximum tightness of 80%, the gap between the yarns is 0.025 mm. To ensure the accuracy of recognition, the gap should occupy at least two pixels in the scanned image. Under this condition, the corresponding resolution is not less than 2,032 dpi. So, in this study, the yarn-dyed fabric images are captured with a resolution of 2,400 dpi.

As shown in Figure 1, the image captured in 2,400 dpi contains 946 pixels and 26.5 warp threads per 1 cm. The wrap density of the fabric in the image is 67.3 threads/inch. Then, we can obtain that each warp yarn occupies about 35.7 pixels, which can ensure recognition accuracy. This calibration will serve as the benchmark for our subsequent recognition of fabric structural parameters. When acquiring fabric images, we should try to avoid the edge of the fabric. Moreover, the surface of the fabric should be clean and flat.

Figure 1

When capturing fabric images, it is difficult to ensure that the warp and weft yarns of the fabric remain completely horizontal and vertical. However, the inclination of the yarns in the fabric image will generally cause a large error in the recognition of fabric parameters. To improve the performance of the recognition, skew correction is required.

In this study, the process of skew correction is shown in Figure 2. We first enhance the fabric image, specifically, (1) convert the acquired RGB image to HSV (hue, saturation, value) space; (2) extract the brightness component V and perform histogram equalization on it. Considering that the interlacing nodes of the fabric are closely related to the brightness, we use the brightness component to detect the tilt angle of the fabric. The used histogram equalization can be represented by: (1) s k = T ( r k ) = L − 1 M N ∑ j = 1 k n j , k = 0 , 1 … , L − 1 , {s}_{k}=T({r}_{k})=\frac{L-1}{MN}\mathop{\sum }\limits_{j=1}^{k}{n}_{j},\hspace{1em}k=0,1\text{…},\hspace{.25em}L-1, where MN is the total number of pixels in the image, L represents the number of gray levels of the image, and n _k is the number of pixels whose grayscale is r _k . Figure 3 presents the effect of the histogram equalization, where Figure 3(a) shows the brightness component corresponding to the fabric image in Figure 4, and Figure 3(b) shows the result enhanced by the histogram equalization. The interlacing nodes in the enhanced image are more prominent.

Figure 2

Figure 3

Figure 4

Then, the OTSU (named after the founder’s name) method is applied to segment interlacing nodes of the fabric. Due to the complex colors in the yarn dyed-fabric, directly using this method for threshold segmentation of the entire image will cause many interlacing nodes to be unrecognized, as shown in Figure 5(a). This study first divides the image into multiple blocks, as shown in Figure 5(b), and then uses the OTSU method to threshold each block. It is stated here that the number of grids is not fixed, but depends on the image size. In this study, each block contains at least 250 pixels (50 px × 50 px). Figure 5(c) presents the segmentation results using the proposed strategy. Compared to Figure 5(a), many missing interlacing nodes are recognized. After obtaining the binary image, the Canny operator is used to detect the edge, and the detection result is shown in Figure 5(d). The promising result demonstrates the effectiveness of the proposed method for interlacing node recognition.

Figure 5

The aforementioned method can detect most of the interlacing nodes of fabric images. Then, two morphological methods, including erosion and dilation, are used to enhance the features of interlacing nodes. When detecting the warp inclination, the kernel of erosion and dilation is (4,1), and when detecting weft inclination, the kernel of erosion and dilation is (1,4). We apply Hough transformation to detect the lines in the binary image. The Hough transformation can detect thousands of straight lines in the image. We filter the straight lines that pass through more interlacing nodes by limiting the angle range and the length of the straight line. The detection results are shown in Figure 6. Finally, we regard the mean value of the inclination angle of the filtered straight line as the inclination angle of the yarns in the image. The proposed method detects that the warp inclination angle of the sample fabric is 2.16° (90 − θ). After obtaining the warp-inclination angle, we rotate the image by 90 − θ, and then we can obtain an image where the warp is vertical, as shown in Figure 7(a).

Figure 6

Figure 7

When the warp yarns are in a vertical state, we use affine transformation to correct the weft-inclination. The warp-corrected image undergoes the aforementioned steps, including threshold segmentation, edge detection, morphological processing (kernel: (1,4)), and Hough transform to obtain the tilt angle. Using the tilt angle, we perform the affine transformation to correct the weft. The weft correction result of the sample fabric is shown in Figure 7(b).

In the yarn image, the yarn axis part reflects more light, showing higher brightness; whereas the reflected light on both sides of the yarn is less, showing lower brightness [28]. The yarns in the yarn-dyed fabric image show a similar rule, from the axis to the gap between the yarns, the gray level gradually decreases. This rule can be used to detect the yarns in the fabric image.

For accurate recognition, we adopt an efficient method to strengthen the gap between the yarns. First, we use a relative total variation (RTV)-based smooth method [29] to eliminate the details of the image. The smooth method can be expressed by: (2) arg min S ∑ p ( S p − I p ) 2 + λ ⋅ D x ( p ) L x ( p ) + ε + D y ( p ) L y ( p ) + ε , \text{arg}\mathop{\text{min}}\limits_{S}\text{}\sum _{p}{({S}_{p}-{I}_{p})}^{\text{2}}+\lambda \cdot \left(\frac{{D}_{x}(p)}{{L}_{x}(p)+\varepsilon }+\frac{{D}_{y}(p)}{{L}_{y}(p)+\varepsilon }\right), where the term (S _p − I _p )² makes the input and the result does not deviate wildly. The effect of removing texture from an image is introduced by a new regularizer (D _x (p)/(L _x (p) + ɛ) + D _y (p)/(L _y (p) + ɛ)), which is called RTV. λ is a weight to control the degree of smoothness and ɛ is a small positive number to avoid division by zero. The division is an element-wise operation. More details can be found in ref. [29]. The smoothing result for the image of yarn-dyed fabric is shown in Figure 8(a). The uneven brightness distribution of the yarn in the fabric image is weakened or eliminated. This result can be used for color segmentation. By dividing the original image I element-wise by the smoothed image Is, we can obtain the gap-enhanced image, as shown in Figure 8(b).

Figure 8

The automatic interlacing node recognition of fabric images can be achieved by using the brightness characteristic information of the images. In this study, all the yarns can be regarded lying as straight stripes in the vertical and the horizontal directions after the skew correction and gridding model are constructed to separate the warp yarns as well as the weft yarns. As mentioned at the beginning of this section, the brightness distribution of the fabric image has an obvious gradient change. The gradient of fabric image intensity arranged in descending order is the axial line of yarns, the marginal zone of yarns, and the transition region between the yarns; in addition, an intensity mutation can be observed in the transition region, so the minimum intensity value can be found by using the projection algorithm in the vertical and the horizontal directions. With this method, the yarns can be detected and the interlacing nodes can be successfully located. Assume that B(i, j) is the brightness value of the pixel in the ith row and jth column in the image. Then the fabric image can be mapped into two independent one-dimensional waves by using the following equations: (3) B ( i ) = ∑ i B ( i , j ) , B(i)=\sum _{i}B(i,j), (4) B ( j ) = ∑ j B ( i , j ) , B(j)=\sum _{j}B(i,j), where B(i) is the accumulated brightness value of ith row pixels and B(j) is the accumulated brightness value of jth column pixel of the fabric image. For the obtained image with enhanced transition area, we first convert it to HSV color space and extract its brightness component V. Then component V is mapped into the two independent one-dimensional waves. The mapping results of the sample fabric image are shown in Figure 8(c) and (d).

It is clearly shown that the valleys of the curve are corresponding to the weft gap position in the fabric image, whereas the peaks represent the axes of the yarns. The case of the warp yarns is similar to that of the weft described above, but the warp yarns lay in the vertical direction. Generally speaking, if B(i) > B(i − 1) and B(i) > B(i + 1) in an ideal smooth curve, B(i) can be regarded as a peak; if B(i) < B(i − 1) and B(i) < B(i + 1), B(i) can be regarded as a valley. However, the fact is that the intensity of the pixel points in the image does not completely follow a descending distribution. It is mentioned before that one yarn occupies about 36 pixels. So we use a 30-pixel pitch slider in the warp and weft directions to search for local minimums, respectively, and the distance between two adjacent local minimums to be greater than 30. The local minimums searched out are considered to be the gap between the yarns in the fabric image, as shown in Figure 8(e). Based on this result, we can achieve automatic measurement of fabric warp density and weft density.

The density of yarn-dyed fabric can be computed with the resolution of the image when the yarns are recognized successfully by using the following equation: (5) D = N ⋅ S dpi M , D=\frac{N\cdot {S}_{\text{dpi}}}{M}, where D denotes the density of yarn-dyed fabric with thread/inch as its unit; M represents the number of pixels among the first and last yarns; N is the number of (warp or weft) yarns in the fabric image; and S _dpi is the resolution, as mentioned before, S _dpi = 2,400. According to this definition, we can obtain the density of the sample fabric shown in Figure 4: D _warp = 79.9 threads/inch, where M = 2,228, N = 74; D _weft = 59.2 threads/inch, where M = 1,580, N = 39.

The node type depends on the relative position of the weft and warp where they interlace. The node is defined as a warp node when the warp is at the top; otherwise, it is defined as a weft node. In this article, the node types are determined by a quadrilateral boundary of interlacing nodes, which is proposed in ref. [30]. The matrix representation of node type is shown in Figure 9(a).

Figure 9

In the yarn-dyed fabric, color information is an important feature. The color feature of each interlacing node is represented by the following equation: (6) V = ( R ¯ , G ¯ , B ¯ ) , V=(\overline{R},\overline{G},\overline{B}), where (7) R ¯ = 1 h w ∑ i = 1 h ∑ j = 1 w r ( i , j ) , \overline{R}=\frac{\text{1}}{hw}\mathop{\sum }\limits_{i=\text{1}}^{h}\mathop{\sum }\limits_{j=\text{1}}^{w}r(i,j), (8) G ¯ = 1 h w ∑ i = 1 h ∑ j = 1 w g ( i , j ) , \overline{G}=\frac{\text{1}}{hw}\mathop{\sum }\limits_{i=\text{1}}^{h}\mathop{\sum }\limits_{j=\text{1}}^{w}g(i,j), (9) B ¯ = 1 h w ∑ i = 1 h ∑ j = 1 w b ( i , j ) , \overline{B}=\frac{\text{1}}{hw}\mathop{\sum }\limits_{i=\text{1}}^{h}\mathop{\sum }\limits_{j=\text{1}}^{w}b(i,j), where h and w represent the height and width of the corresponding interlacing node, respectively. r(i, j), g(i, j), and b(i, j) represent the value of red, green, and blue component at point (i, j). In other words, we use the average value of R, G, and B of all pixels in the interlacing node to represent it. However, the colors at the junctions between the yarns affect each other, making the colors in some interlacing nodes have outliers. Assume that the color of each pixel in the interlacing node obeys a 4D Gaussian distribution, expressed as c(r, g, b) ∼ N (C|μ,∑). The probability density function can be expressed as: (10) f ( c ) = 1 ( 2 π ) 3 ∑ 1/2 exp − ( c − μ ) T ∑ − 1 ( c − μ ) 2 , f(c)=\frac{1}{{(\sqrt{2\pi })}^{3}{\left|\sum \right|}^{\text{1/2}}}\text{exp}\left(-\frac{{(c-\mu )}^{T}{\left(\sum \right)}^{-1}(c-\mu )}{2}\right), where Σ is the covariance matrix, and μ is the matrix consisting of the mean value of the three components of R, G, and B. We select those pixels with a 95% confidence level as statistical samples to represent the color feature of each interlacing code.

The FCM clustering method is proposed to classify the nodes with the color features extracted from the nodes. And it is an unsupervised method, which attempts to minimize the objective function by organizing the data into different clusters. We follow the method proposed in ref. [26] for node classification. However, the number of yarn colors in the yarn-dyed fabric should be known first. For automatic detection, Pan et al. [26] introduced the cluster validity criterion V _MPC [31] to find the optimal cluster number. V _MPC indices of the sample yarn-dyed fabric are shown in Table 1. We can easily find that the sample fabric does contain five colors of yarn.

FCM (c = 5) is used to classify the nodes with the color feature. The color of the nodes is then replaced by the corresponding cluster center. The boundary pixels are removed from the image and the nodes are expressed by the color rectangles of the same size. As shown in Figure 9(b), the color pattern can be obtained by using this method. We can also obtain the layout of color yarns by combining the node type and color pattern.

For a yarn-dyed sample, first the scanner is used to obtain its image. The inclination of the yarns in the fabric image will generally cause errors in the recognition of fabric parameters. Therefore, the method introduced in Section 3 is proposed to correct the warp and weft yarns. For accurate detection of the yarns in the image, a brightness-projection method is used to locate the gaps between the yarns, and thus recognize the layout of yarns. The results can be applied to measure the density of the yarn-dyed fabric. The types of interweaving nodes are recognized by using the method proposed in ref. [30]. FCM is proposed to classify the nodes with the color features extracted from the nodes. Combining the clustering result and weave pattern, we can obtain the color pattern of the yarn-dyed fabric.

Table 2 and Figure 10 present the actual parameters and detection results of three yarn-dyed fabric samples. The promising results demonstrate the effectiveness of the proposed method for the recognition of yarn-dyed fabric.

Figure 10

To demonstrate the superiority of the proposed method, we compare some current methods on this topic including those of Pan et al. [26], Guo [5], Xin et al. [27], Iqbal Hussain et al. [2], and Meng et al. [1]. In this experiment, we implemented the aforementioned method following the authors’ paper and then tested it on 100 samples of yarn-dyed fabrics. Here, error rate [1] is employed to evaluate the recognition performance of contrasting methods. The experimental results are presented in Table 3. The proposed method achieves superior performance of 2.1%, 1.2%, 1.4%, and 0.6% on various metrics, outperforming other comparative methods. The method of Iqbal Hussain et al. [2] is the best performing method among the compared methods, however, our method reduces the error rate by about 2%. The experimental results show that the proposed method has a certain improvement in the identification of yarn-dyed fabrics, which demonstrates the superiority of the proposed method.

We also use the method proposed in ref. [1] to recognize the yarn layout of the yarn-dyed fabrics, and the results are presented in Figure 11. The poor results are mainly caused by two reasons: (1) The model was trained from a small dataset, which leads to its low generalization; (2) the image resolution is different from the original. These results prove that in the case of limited data, low-level feature-based methods are more effective than learning-based methods.

Figure 11

Aiming at the recognition of the structural parameters of yarn-dyed fabric, an effective method is proposed based on the previous work, especially refs [1,26,30]. In the recognition system, the fabric images are captured by a scanner with a resolution of 2,400 dpi. The inclination angles of the warps and wefts of the fabric are detected by Hough transform. Then we use rotation transformation and affine transformation to correct the fabric image, respectively. A smoothing method is used to enhance the yarns and the gaps between the yarns to improve the accuracy of the yarn location. Using the detection results, the density of yarn-dyed fabric can be calculated by equation (5). The edge information of the four directions of the interweaving node is the main basis for judging whether it is a warp node or a weft node. The number of yarn colors obtained from cluster validity analysis and the extracted color features are used to classify the nodes with FCM algorithm. Based on the clustering results, the color pattern of yarn-dyed fabric can be obtained.

The proposed method has some limitations. For jacquard fabrics or multi-layer fabrics, our method may not work. However, it still can be used to measure the density of the fabric.