Prediction of Standard Time of the Sewing Process using a Support Vector Machine with Particle Swarm Optimization

A machine learning algorithm called SVM based on statistical learning theory and SRM principle performs well for classification and regression properties [21]. Thus, SVM can be divided into support-vector classification machine and support-vector regression machine. The former usually deals with classification problems, and the latter is used for prediction. The purpose of support-vector regression machine is to analyze the function y_i = f(x_i) between the input and output according to the given training sample (x₁, y₁), (x₂, y₂) … (x_n, y_n). Assuming the function is f : x → f(x), y′ = f (x), the output y′ is the regression value based on the regression function. It is clear that standard time forecasting belongs to the typical nonlinear problem. The input variables are mapped into a high-dimensional linear feature space through a nonlinear transformation φ (x). The regression function in high-dimensional space, which represents the relationship between standard time of the sewing process and input variables, can be defined as follows: f(x) = ω · φ (x) + b, where ω is the weight vector and b is bias. The values of ω and b can be obtained by using the following minimization equation, where ξ_i and ξi*\xi _i^* are relaxation factors, and ‖ω‖² represents the confidence range to smooth the fitting function and strengthen the promotion ability of fitting function. The constant c is a penalty coefficient when the training data are out of the channel, and it can penalize the error by determining the trade-off between the training error and the model complexity. The larger value of c represents the hard-margin regression function, while the smaller value indicates that the estimation function allowed deviating from ɛ with a lower cost. If the regression function f(x) can estimate all the training sample points, the minimization problem could be transformed into the following optimization problem: (1)min[12‖ω‖2+c∑i=1n(ξi+ξi*)]s.t. yi-ω⋅φ(xi)-b≤ε+ξiω⋅φ(xi)+b-yi≤ε+ξi*ξi≥0,ξi*≥0, i=1,2,…,n\matrix{ {\min \left[ {{1 \over 2}{{\left\| \omega \right\|}^2} + c\sum\nolimits_{i = 1}^n {\left( {{\xi _i} + \xi _i^*} \right)} } \right]} \hfill \cr {s.t.\,{y_i} - \omega \cdot \varphi \left( {{x_i}} \right) - b \le \varepsilon + {\xi _i}} \hfill \cr {\omega \cdot \varphi \left( {{x_i}} \right) + b - {y_i} \le \varepsilon + \xi _i^*} \hfill \cr {{\xi _i} \ge 0,\xi _i^* \ge 0,\,i = 1,2, \ldots ,n} \hfill \cr }

Since Eq. (1) is a convex optimization problem, the LaGrange multipliers will be introduced to obtain the LaGrange function. Then, the original problem could be transformed into the corresponding dual problem and the regression function can be obtained as follows: (2)f(x)=∑i=1n(ai-ai*)K(xi,x)+b*f\left( x \right) = \sum\nolimits_{i = 1}^n {\left( {{a_i} - a_i^*} \right)K\left( {{x_i},x} \right) + {b^*}} where a_i and ai*a_i^* are the LaGrange multipliers, which are automatically selected by the SVM algorithm during development of the model. In Eq. (2), K (x_i, x) is a kernel function, which attempts to fit a tube in a higher dimensional space. There are four commonly used kernel functions, i.e., radial basis kernel, sigmoid kernel, polynomial kernel, and linear kernel. Choosing different kernel functions can generate different SVM models. In this paper, the radial basis kernel was utilized: K(xi,x)=exp(-‖x-xi‖22γ2)K\left( {{x_i},x} \right) = {exp}\left( { - {{{{\left\| {x - {x_i}} \right\|}^2}} \over {2{\gamma ^2}}}} \right) , where the free parameter γ is the kernel parameter.

The SVM model has a good ability to solve small-sample, high-dimensional, and nonlinear problems. However, the choice of kernel function parameter γ and the penalty parameters c of the SVM model have important influences on the accuracy of the SVM model. Only by constantly adjusting the model parameters to achieve the best combination of model parameters can the SVM machine learning ability and regression prediction effect be improved. Therefore, the penalty parameters c and kernel parameters γ should be optimized. The traditional methods of SVM parameter selection are generally the cross validation method and grid search method. These two methods have some limitations: low efficiency, low precision, and the fact that the search parameters cannot be optimized. PSO is a new evolutionary algorithm developed in recent years, and it was inspired from the feeding behavior characteristic of a bird flock, which is used for solving the optimization problems [22]. In the PSO algorithm, a group of particles was initialized in the feasible solution space, each potential solution to the problem was treated as a particle, and the common feature of the particle is represented by position, speed, and fitness value. The fitness value is calculated by the fitness function, which is worthy of the pros and cons of the said particle. In the D-dimensional space, the space vector X_i = (X_i1, X_i2, …, X_iD)^T, is represented as the ith particle, where i = 1, 2, …, n (n population quantity) and X_i is the position of the ith particle and a possible solution. After finding the local and globally optimal solutions, the particle would update its speed and new position according to the following equation: (3)Vidk+1=wVidk+c1r1(Pidk-Xidk)+c2r2(Pgdk-Xidk)V_{id}^{k + 1} = wV_{id}^k + {c_1}{r_1}\left( {P_{id}^k - X_{id}^k} \right) + {c_2}{r_2}\left( {P_{gd}^k - X_{id}^k} \right)(4)Xidk+1=Xidk+Vidk+1X_{id}^{k + 1} = X_{id}^k + V_{id}^{k + 1}

In the formula, k is the kth iteration; V_i = (V_i1, V_i2, …, V_iD) is the velocity of the ith particle, and P_i = (P_i1, P_i2, …, P_iD) is the optimal position of this particle. The optimal swarm position is P_g = (P_g1, P_g2, …, P_gD) Under the condition of the ith particle at the kth iteration, Vidk+1V_{id}^{k + 1} and Xidk+1X_{id}^{k + 1} are the dth speed component and location, respectively. Parameters r₁ and r₂ are the random number, the range is 0–1, c₁ and c₂ are the learning factors between 0 and 2, and w is the inertial weight of the PSO algorithm. To prevent the particle disengaging from the search space in the search process, V_id ∈ [−V_max, V_max], where V_max is the maximum flying speed. To speed up the convergence of the algorithm, w is linearly reduced as the iteration progresses: (5)w=wmax-iter(wmax-wmin)/itermaxw = {w_{{max}}} - iter\left( {{w_{{max}}} - {w_{{min}}}} \right)/{ite}{{r}_{{max}}} where iter and iter_max are the current and maximum iteration times, respectively; and w_max and w_min are the maximum and minimum inertial weights, respectively [19].

The process of using PSO for parameter optimization is as follows:

The population is initialized. The population size, the maximum number of iterations of the population, the penalty factor c, and the optimization range of the kernel parameter γ are set. The learning factors c₁ and c₂ are adopted by the linear learning strategy. The inertial weight w is adopted by the linear decreasing strategy.

The raw data were adapted from ZS Company, which is a privately held family owned clothing company based in China. The company was founded in 1989, owning four fashion brands, and has accumulated a great amount of sewing standard time data during its multiple years of production. After cleaning the data to remove incomplete records, the final data set consists of 105 sewing processes, including standard time and influencing factors, as shown in Table 2.

In order to eliminate the influence of data dimensions on the prediction results, the original data need to be preprocessed. The dimensionality reduction method used in this paper is the minimum–maximum (min–max) method, and the calculation formula is expressed as follows: (6)xi'=(xi−xmin)(xmax−xmin)x_i^\prime = {\raise0.7ex\hbox{${\left( {{x_i} - {x_{{min}}}} \right)}$} \!\mathord{\left/{\vphantom {{\left( {{x_i} - {x_{{min}}}} \right)} {\left( {{x_{{max}}} - {x_{{min}}}} \right)}}}\right.}\!\lower0.7ex\hbox{${\left( {{x_{{max}}} - {x_{{min}}}} \right)}$}} where x_i is the ith item in the sequence; x_min is the minimum value of the sequence; and x_max is the maximum value of the sequence. The significance of min–max dimensioning is to project the whole sequence onto the [0,1] interval in the process of dimensioning the sequence, where the projection process does not change the scale relationship between the sequence items.

Influencing factors are input data for machine learning algorithms. During the sewing process, as mentioned above, many factors affect the standard time, such as sewing length, stitch density, bending stiffness, fabric weight, production quantity, drape coefficient, and length of service. Before making the prediction, the main influencing factors of standard time in the sewing process were identified by grey correlation analysis. Grey correlation analysis aims to quantitatively describe the relationship or development trend between variables according to the correlation degree. The reference sequence reflecting the system's behavior characteristics and the comparison sequence affecting the system's behavior are both supposed to be determined. The data sequence that reflects the behavior characteristics of the system is called reference sequence. The data sequence composed of factors that affect the system's behavior is called comparison sequence [30]. If the correlation degree between the comparison sequence and the reference sequence is large (correlation degree > 0.6), the relationship between the two is considered to be close; on the other hand, if the correlation degree is small (correlation degree < 0.5), the relationship between the two is distant. This is used as the basis to judge the correlation. The grey correlation coefficient ξ(ij) of the reference sequence and the comparison sequence is computed and given as follows: (7)ξ(ij)=minimin1≪j≪n|x0(j)-xi(j)|+ρmaximax1≪j≪n|x0(j)-xi(j)||x0(j)-xi(j)|+ρmaximax1≪j≪n|x0(j)-xi(j)|\xi \left( {ij} \right) = {{\mathop {{min}}\limits_i \mathop {{min}}\limits_{1 \ll j \ll n} \left| {{x_0}\left( j \right) - {x_i}\left( j \right)} \right| + \rho \mathop {{max}}\limits_i \mathop {{max}}\limits_{1 \ll j \ll n} \left| {{x_0}\left( j \right) - {x_i}\left( j \right)} \right|} \over {\left| {{x_0}\left( j \right) - {x_i}\left( j \right)} \right| + \rho \mathop {{max}}\limits_i \mathop {{max}}\limits_{1 \ll j \ll n} \left| {{x_0}\left( j \right) - {x_i}\left( j \right)} \right|}}

When we input the ρ-identification coefficient, ordinary ρ gets 0.5. The correlation degree r_0i formula is as follows: (8)r0i=1n∑j=1nξ(ij), i=1,2,3,4,5,6,7{r_{0i}} = {1 \over n}\sum\nolimits_{j = 1}^n {\xi \left( {ij} \right),\,\,i = 1,2,3,4,5,6,7} where r_0i is the grey correlation degree of the sequence x_i to the reference sequence x₀. The closer r_0i is to 1, the better is the correlation.

The standard time in the sewing process was taken as the reference sequence, and the influencing factors are the comparison sequences. MATLAB can be used to calculate the related grey correlation degree. According to Table 3, the influencing factor with the maximum correlation degree for the standard time of the sewing process is the sewing length (0.823), and the influencing factor with the minimum correlation degree is the seam density (0.613). When the grey correlation degree is 0.8–1.0, there is a high degree of correlation between the reference sequence and the comparison sequence; when the grey correlation degree is 0.6–0.8, the reference sequence and the comparison sequence have a relatively high degree of correlation; when the grey correlation degree is <0.5, the correlation between the reference sequence and the comparison sequence is weak. Because none of the influencing factors is <0.6, this paper selects the above seven factors as the input vector of SVM.

A total of 105 samples were collected in the construction of the standard time prediction model of the sewing process. In this case study, 88 samples were selected randomly as training samples, and 17 samples were used as testing samples to train and test the PSO-SVM model. The MATLAB and LIBSVM toolbox were used to implement the SVM model according to the construction process. When the PSO-SVM program was written, the initial parameters of the model were set. The standard PSO with 20 particles (SIZEPOP = 20) and 200 iterations was used (MAXGEN = 200). The value range of γ was (0.01, 1,000) and c was (0.1, 150). In PSO, the learning factor was a random number between 0 and 2. In this paper, the learning factors were c₁ = c₂ = 1.5. The machine environment was an i5-2430M central processing unit (CPU) with a frequency of 3.90 GHz and a memory of 4.0 GB, running on the Windows 7 operating system. After initialization, the calculation program was processed to read the sample data of the training set. The kernel parameter γ = 0.01 and penalty parameter c =101 of SVM were obtained.

For the sewing process, the PSO-SVM model with radial basis kernel function is introduced to predict the standard time. MSE and squared correlation coefficient (R²) are introduced as performance criteria for prediction assessment. The comparison between the actual and predicted values in the training set is shown in Figure 2. In Figure 2, x-axis represents the sample size of the training set, and y-axis the standard time of the sewing process. Then, the regression prediction was made for the test data of randomly selected test samples. The comparison between the actual value and the predicted value is shown in Figure 3. As shown in Table 4, in the training experiment, the R² criterion is 0.917. In the testing experiment, the R² is 0.852. Irrespective of whether the process involved is the training data experiment or testing data checking, the prediction model of the standard time of the sewing process based on PSO-SVM optimization algorithm achieves higher prediction performance. Therefore, the prediction model of PSO-SVM has excellent generalization ability and can obtain a scientific prediction result for standard time of the sewing process.

Figure 2

Figure 3

In order to further validate the performance of the model, we also presented a BP neural network to make a prediction experiment. The forecasting result is shown in Figure 4. As shown in Table 5, MSE of the BP neural network is much higher than that of SVM, and R² of SVM is closer to 1. It is obvious that the prediction result of SVM is much better than that of the BP neural network. Thus, it is proven that SVM can be effectively applied to the prediction problem of a small sample while maintaining sufficient accuracy.

Figure 4

In the SVM theory, determining the model parameters such as penalty parameter c and kernel function parameter γ is a very important step in the process of model establishment. The conventional parameter search method is the cross validation method for SVM. The basic idea of cross validation is that we usually do not use all the data sets for training, but use part of it (this part does not participate in the training) to test the parameters generated by the training set, and relatively objectively judge the parameters of the data outside the training set. For the PSO-SVM optimization algorithm, parameter optimization is obtained by judging the individual extremum and global optimal solution of the particle, as well as adjusting the velocity and position of the particle. Training data is conducted with different model parameters and verified on the validation set to determine the most appropriate model parameters. Table 6 shows the model performance based on different parameter search methods. As demonstrated in Table 6 from the MSE results, it could be observed that the SVM-derived result with the model parameters selected by PSO has better performance than those of the cross validation methods. Therefore, it can be concluded that the PSO-SVM method has a better model prediction capability than the ordinary SVM method based on cross validation.