Case application data research of traditional ink art elements in packaging design

The function of the ink-wash image preprocessing system is that, after processing input ink-wash image, we get an output which is the processed result data called primitives. In fact, the output primitives are organically combined into ‘objects’ of this module. The area of interest in the ink image is separated, and then it is segmented, fused, then skeleton extraction and other processing are performed, and finally we get the primitive information as the output. The ink and wash image preprocessing system is the first processing module designed in this paper. If the ink wash image preprocessing system to the stylised rendering system is regarded as an ink wash image data processing process, then the preprocessing system and every other subsystem again are associated input and output data [11]. Ink image preprocessing is the premise of the design of ink painting artistic style. Only on the basis of good ink image preprocessing specific feature analysis and feature extraction can be carried out effectively. Therefore, it is very important to choose a stable preprocessing algorithm. First of all, for the result of ink image processing, as these primitive information will be transmitted to the style learning system, they will be called input samples in the style learning system. It is stipulated here that in the ink-wash image preprocessing subsystem, the results of the ink-wash image processing are collectively referred to as primitives [12].

The skeleton information obtained from the skeleton extraction of the texture describes the extension direction of the primitive pixels to a certain extent. The length of the skeleton in this system is normalised to [0, 1]; the position coordinate (x, y) of any point on the skeleton in the primitive texture area is the u function of the distance travelled along the skeleton from the skeleton zero point, here it is called the skeleton description function S (u) [13]. The skeleton description function can be defined as: (1) S(u)=P(u⋅(N−1)) S\left(u \right) = P(u \cdot (N - 1)) where N is the total number of pixels in the skeleton; P(i) represents the position coordinates of the i-th pixel of the skeleton in the texture area, i ∈ [0, N].

The system not only needs to extract the various primitives of the original ink image, but also organically combine the discrete primitives into different ‘objects’.

An object is a collection of primitives that are relatively close in the spatial and colour domains. From the perspective of the whole system, combining primitives into objects is the process of extracting objects of different ink styles from ink works [15]. Therefore, the ink image segmentation algorithm in this paper consists of two parts namely region growing and object extraction. Region growing is a process of aggregating pixels or seed regions into large regions according to pre-defined criteria; the basic method is to start with a set of ‘seed’ points and attach adjacent pixels with similar properties to the seed to each seed of the growing region, until no pixels meet the conditions for joining a region. The most important part of the region growing method is the selection algorithm of seed points and the similarity measure function between regions. Here, the description of the two parts selected for this topic is given first, and then the details of the region growing algorithm process are described in general [16].

The inter-region similarity measure employed during growth is based on the region-averaged colour distance. Here, the colour distance is defined in the a*b colour space, not the RGB colour space. Because the visual perception of colour in the RGB colour space is non-linear. The a*b colour space is suitable for the representation and calculation of all light source colours or object colours. It is a space that reflects the uniformity of object colours in human perception. The colour space distance obtained in this space is more in line with human visual perception; Perceptual properties of colour differences. Since the ink images processed by this system are mainly stored in RGB colour format, this paper needs to know the conversion formula between the two. To make the conversion easier, this paper first introduces the CIEXYZ space as the transition space. The formula for converting RGB colour space to CIEXYZ space is as Eq. (2): (2) (XYZ)=100⋅(0.41240.35760.18050.21260.71520.07220.01930.11920.9505)⋅(f(R/255)f(G/255)f(B/255)) \left({\matrix{X \cr Y \cr Z \cr}} \right) = 100 \cdot \left({\matrix{{0.4124} & {0.3576} & {0.1805} \cr {0.2126} & {0.7152} & {0.0722} \cr {0.0193} & {0.1192} & {0.9505} \cr}} \right) \cdot \left({\matrix{{f(R/255)} \cr {f(G/255)} \cr {f(B/255)} \cr}} \right)

The conversion formula from CIEXYZ space to a*b space is as follows (3) L=116⋅g(Y/Y0)−16 L = 116 \cdot g\left({Y/{Y_0}} \right) - 16 (4) a=500⋅g(X/X0)−g(Y/Y0) a = 500 \cdot g\left({X/{X_0}} \right) - g\left({Y/{Y_0}} \right) (5) b=200⋅g(Y/Y0)−g(Z/Z0) b = 200 \cdot g\left({Y/{Y_0}} \right) - g\left({Z/{Z_0}} \right) X, Y, Z are the tristimulus values of the sample colours, X₀, Y₀, Z₀ are the tristimulus values of the CIE standard illuminator, and are constant under the condition of specifying the standard illuminator. Here, considering D65 light source lighting conditions, we get (6) g(t)={t1/3t>0.0088567.787t+16/116others g(t) = \left\{{\matrix{{{t^{1/3}}} \hfill & {t > 0.008856} \hfill \cr {7.787t + 16/116} \hfill & {others} \hfill \cr}} \right.

In the a*b space, the colour difference between the two colours (L₁, a₁, b₁) and (L₂, a₂, b₂) is defined as the Euclidean distance between them in this space, as shown in Eq. (7) (7) Dlab=(L1−L2)2+(a1−a2)2+(b1−b2)2 {D_{lab}} = \sqrt {{{\left({{L_1} - {L_2}} \right)}^2} + {{\left({{a_1} - {a_2}} \right)}^2} + {{\left({{b_1} - {b_2}} \right)}^2}}

Skeletons, also known as regional centrelines, are a way to express the structure of a target shape. From the definition of primitives, we can see that the skeleton largely expresses the extension direction of a primitive texture, and at the technical level, it also reflects the path and direction of the painter's strokes when depicting objects [17]. In addition, the texture features of the primitive itself are usually generated along this path, and the average width of the texture that the system needs is to be calculated based on the skeleton information. In general, skeletons have three main properties: continuity, minimum width of 1 and centrality. The traditional technique for extracting the skeleton of a region is the mid-axis transformation. For the skeleton extraction algorithm, first mark the known target point as 1 and the background point as 0; consider the 8-neighbourhood with the boundary point as the centre, denote the centre point as P, and the 8 points of the neighbourhood circle the centre point counterclockwise. They are marked as P₀, P₁, P₂, P₃, P₄, P₅, P₆, P₇ with P₀ to the right of P. The boundary points on the entire area set in the first step should be the boundary points that meet the conditions of 2 ≤ N_s(P) ≤ 6, where N_s(P) is the number of non-zero neighbours in the 8 fields of P, that is, as shown in Eq. (8) (8) Ns(P)=∑i=07Pi {N_s}(P) = \sum\limits_{i = 0}^7 {P_i} S(P) is called the intersection number of point P, which is the number of changes from 0 to 1 when rotating in order, that is, as shown in Eq. (9) (9) S(P)=∑k=08(Pk−Pk−1) S(P) = \sum\limits_{k = 0}^8 \left({{P_k} - {P_{k - 1}}} \right)

After checking all boundary points, delete all marked points, and repeat the above steps until no boundary points are marked. The skeletons generated by this method have a disadvantage and are prone to glitches, i.e., short skeleton branches due to unsmooth region boundaries [18]. The glitches interfere with the analysis process in this paper and should be removed. Using the burr culling algorithm, the generated skeleton can be very smooth.

In the skeleton calculation, this paper obtains the shape description of the pixel area, which guarantees the extraction of other information of the primitives in this section. This paper will describe the extraction algorithm of each component data of the primitive [19]. The geometric area refers to the size of the smallest rectangular bounding box representing the pixel area of the primitive itself and its coordinate position in the original image. Since the index number of the segmentation area to which each pixel in the original image belongs has been marked in the ink image segmentation stage, it is natural to get the pixel coordinates of the lower left corner of a certain pixel area when scanning the original image from bottom to top and from left to right and top right position. Here, the lower left corner is marked as (x₀, y₀), the upper right corner coordinates as (x₁, y₁) and the primitive collection area is Rg. It contains four components, which respectively represent the position coordinates and width and height dimensions of the geometric area in the original image. The formulas used for calculation are as follows: (10) Rg{Rg⋅x=x0Rg⋅y=y0 Rg\left\{{\matrix{{Rg \cdot x = {x_0}} \hfill \cr {Rg \cdot y = {y_0}} \hfill \cr}} \right. (11) Rg1{Rg⋅width=x1−x0Rg⋅height=y1−y0 Rg1\left\{{\matrix{{Rg \cdot width = {x_1} - {x_0}} \hfill \cr {Rg \cdot height = {y_1} - {y_0}} \hfill \cr}} \right.

Primitive ink density represents the average grey level of the primitive texture. In this system, the average grey level of the primitives is normalised to [0, 1] for the convenience of ink colour analysis.

The main colour of the primitive is actually the average colour information of the statistical primitive. Since this information will be used for colour recovery of greyscale primitive textures, this paper should consider extracting the average colour information of the primitives instead of simple RGB colour values. An optional algorithm used here to describe colour information is the HSI colour model [20]. The HSI value is a triple (H, S, I), where H, represents the hue of a colour pixel, S represents the saturation and I represents the brightness (also known as greyscale). Considering that the primitive texture pixels only retain the brightness value after greyscale, the system only needs to calculate the H value and S value of the average colour. In fact, this process is to convert the average RGB value calculated in the previous step into an HSI value, and the replacement formulas are shown in Eqs (12) and (13). (12) H{θB≤G360−θB>G H\left\{{\matrix{\theta \hfill & {B \le G} \hfill \cr {360 - \theta} \hfill & {B > G} \hfill \cr}} \right. (13) θ=arccos(12[(R−G)+(R−B)][(R−G)2+(R−G)(G−B)]12) \theta = \arccos \left({{{{1 \over 2}\left[ {(R - G) + (R - B)} \right]} \over {{{\left[ {{{(R - G)}^2} + (R - G)(G - B)} \right]}^{{1 \over 2}}}}}} \right)

Where R, G, B are the red, green and blue components, respectively. The value of saturation S is given by Eq. (14) (14) S=1−3(R+G+B)min(R,G,B) S = 1 - {3 \over {(R + G + B)}}\min (R,G,B)

In the above three evaluation formulas, if the R, G, B value is normalised to be in the range of [0, 1], then the calculated S value is also in the range of [0, 1]; and the obtained H value will be also in the range of [0, 1]. The HIS model represents the hue angle, which can be normalised to [0, 1] only by setting it at 360.

The average width of the primitive is the average of the lengths of the vertical segments at all points on the skeleton line (the vertical segments are marked by red lines). The calculation formula is Eq. (15): (15) Wavg=∫01WP(u)du {W_{avg}} = \int_0^1 {W_P}(u)du

Among them, W_avg is the average width of the primitive, and W_P(u) is the length of the vertical line segment at the point at u. Considering the discreteness of the real points on the skeleton, the Eq. (15) can be rewritten as the discrete sum Eq. (16) (16) Wavg=1N∑0N−1WP(i) {W_{avg}} = {1 \over N}\sum\limits_0^{N - 1} {W_P}(i) where N is the number of skeleton points, and w_p(i) is the length of the vertical line segment at the i point on the skeleton line.

The outer contour vectorisation algorithm of ink images is also applicable to the inner contour. By observing the inner contour obtained after edge detection, it is found that the trend of the inner contour is obviously smooth, and the length of the inner contour is smaller than that of the outer contour. Therefore, the outer contour vectorisation algorithm is applied to the inner contour. When contouring, two principles are followed in threshold selection: one is to choose a larger angle threshold and the other is to choose a smaller sampling interval [23, 24]. Referring to the outer contour vectorisation algorithm, each line array of the inner contour of the ink image is processed separately, and the vectorised coordinates are stored in the array.

Contour vectorisation greatly reduces the number of pixel coordinates, obtains a vectorised mountain contour, and stores the stationary point coordinates in an array. In this section, an algorithm is designed to translate, rotate and connect the stagnation point, so that a mountain contour line evolves from a line to a strip. Let the width of the spline be d, the coordinate of the stagnant point of the vectorised contour is q(x, y), then the angle between the two sides connected with the stagnation point is θ (0 < θ < ɛ), that is, the sampling threshold angle.

Description of spline drawing algorithm: Traverse the pixel coordinates p(x_i, y_i) on the vectorised contour line from the starting point, and p(x_i, y_i) calculates the distance to the stagnant point q(x, y). If d_i continues to traverse the contour line: if d_i ≠ d, according to the coordinate rotation transformation formula, convert the coordinates of d_i = d, rotate the point (x_i, y_i) and the obtained coordinate point is rounded and denoted as θ2 {\theta \over 2} , then the p(s, t) point coordinate is the coordinate on the p angle bisector. θ rotates around the q(x, y) coordinate to get the point p^′ (s, t), the calculation formula is shown in Eq. (17) (17) {s=(xi−x)cos(θ)−(yi−y)*sin(θ)+xt=(xi−x)sin(θ)+(yi−y)*cos(θ)+y \left\{{\matrix{{s = \left({{x_i} - x} \right)\cos (\theta) - \left({{y_i} - y} \right)*\sin (\theta) + x} \cr {t = \left({{x_i} - x} \right)\sin (\theta) + \left({{y_i} - y} \right)*\cos (\theta) + y} \cr}} \right.

After calculating the mirror point of p^′ (s, t) about the key point q(x, y), p^″ (s, t) is obtained, then the point p^″ is also a point on the angle bisector of the vertex q(x, y).

The purpose of colour segmentation of ink and wash images is to determine the information of the sky, trees, waterfalls and people in the image, and to finally find the area that reflects the outline of the ink and wash image. Based on the Euclidean distance formula mentioned previously, according to the calculation of the colour difference of two pixels, all the pixels whose colour difference ΔE is within a certain range are clustered, and the formula is shown in Eq. (18). (18) E=∑1K∑(Ly,ay,by)(Lx−Ly)2−(ax−ay)2−(bx−by)2 E = \sum\limits_1^K \sum\limits_{({L_y},{a_y},{b_y})} \sqrt {{{({L_x} - {L_y})}^2} - {{({a_x} - {a_y})}^2} - {{({b_x} - {b_y})}^2}}

When E approaches 0, pixel x and pixel y can be regarded as a class.

Greyscale the ink image after colour segmentation to obtain a greyscale image of the ink image. However, although the ink image in the greyscale image preserves the internal mountains and veins of the ink image, it also preserves information such as trees and rivers. These trivial information will interfere with the extraction of the inner contour of the ink image. The purpose of filtering the image is to reduce the interference of useless information such as trees and rivers on the internal vein texture of the ink image, and to highlight the edges of the internal concave and convex texture of the ink image for easy extraction. The filtering of ink images often starts from the spatial domain, rather than the frequency domain filtering commonly used for signals [25]. Spatial domain filtering is a very important application in ink image processing, which is of great help in reducing noise in ink images. The greyscale image preprocessing of the ink image is filtered by calculating the edge gradient features of the ink image, and the interference is filtered while maintaining the original edge. Its filter formula is shown in Eq. (19) (19) {∂I∂t=div[c(ΔI)]I(x,y,0)=I0(x,y) \left\{{\matrix{{{{\partial I} \over {\partial t}} = div\left[ {c(\Delta I)} \right]} \hfill \cr {I(x,y,0) = {I_0}(x,y)} \hfill \cr}} \right. where I₀ is the original ink image, I(x, y, t) is the evolution function of time and ΔI is the gradient magnitude function. In the ink greyscale image, the inner concave and convex edges of the ink image are closely related to the gradient, so the positioning of the concave and convex boundary lines needs to be realised by calculating the gradient. c(·) is a filter control function, and it is also a function of the edge gradient amplitude, which can be regarded as a test of the smoothness of the ink image. The two filter control functions of the PM model are shown in Eqs (20) and (21). (20) c(x)=e−(x/K)2 c(x) = {e^{- {{(x/K)}^2}}} (21) c(x)=11+(x/K)2 c(x) = {1 \over {1 + {{(x/K)}^2}}}

The gradient magnitude of the boundary taken by x is represented by ΔI, where K is the diffusion function, which is usually a constant. Then the edge filter control function transforms to get the following Eqs (22) and (23) (22) c(ΔI)=exp[−(ΔIK)2] c(\Delta I) = \exp \left[ {- {{\left({{{\Delta I} \over K}} \right)}^2}} \right] (23) c(ΔI)=11+(ΔI/K)2 c(\Delta I) = {1 \over {1 + {{\left({\Delta I/K} \right)}^2}}}

According to the mathematical relationship, the boundary gradient magnitude function ΔI of the ink image is represented by the Eq. (24) gradient function (24) |ΔI|=1+(|grad(J)|/K∧2 \left| {\Delta I} \right| = 1 + (\left| {grad(J)} \right|/{K^ \wedge}2

Through the above analysis of the principle of the PM model, the sharpening spatial filter that uses the gradient magnitude function as the control function of the PM model will filter the part with large gradient changes in the ink image, which is obviously contrary to preserving the gradient in the greyscale image. The original intention is of clearly changing lines. Since it is necessary to suppress the filtering in the gradient direction, the filtering control function c(ΔI) is taken as the reciprocal of the boundary gradient magnitude function (|ΔI|) according to the characteristics of the edge and the vertical gradient. In this way, the filtering in the gradient direction in the ink image will be suppressed. The filter control function along the edge direction of the ink image is obtained, as shown in Eq. (25) (25) {c(ΔI)=1|ΔI|c(ΔI)=1/[1+|grad(J)|/K)Λ2] \left\{{\matrix{{c\left({\Delta I} \right) = {1 \over {\left| {\Delta I} \right|}}} \hfill \cr {c\left({\Delta I} \right) = 1/\left[ {1 + \left| {grad(J)} \right|/K{)^\Lambda}2} \right]} \hfill \cr}} \right.

The gradient magnitude (ΔI) is regarded as a detection function of gradient changes in greyscale ink images. c(ΔI) is a non-negative function of the gradient magnitude |ΔI| of ink images. As a function of controlling the filtering of ink and wash images, c(ΔI) is very critical to filter the edges that appear in ink and wash images. The internal texture of the ink image is highlighted after filtering by the PM model, which is convenient for edge detection of the internal texture of the ink image.

In ink image processing, bilinear interpolation algorithm completes the enlargement or reduction of the original ink image by generating new pixels. Adjust the size of the painter's strokes through bilinear interpolation, so that the real ink strokes can match the width of the spline model. Using bilinear interpolation to calculate the pixel value of a point in the target ink image is to find the 2*2 adjacent pixels of the point in the source ink image, and then use the pixel weighted average of the 2*2 area to obtain the pixel value of a point in the target ink image. The formulas for calculating the bilinear interpolation in the x direction are as follows (26) f(R1)≈x2−xx2−x1f(Q11)+x−x1x2−x1f(Q21) f\left({{R_1}} \right) \approx {{{x_2} - x} \over {{x_2} - {x_1}}}f({Q_{11}}) + {{x - {x_1}} \over {{x_2} - {x_1}}}f\left({{Q_{21}}} \right) (27) f(R2)≈x2−xx2−x1f(Q12)+x−x1x2−x1f(Q22) f\left({{R_2}} \right) \approx {{{x_2} - x} \over {{x_2} - {x_1}}}f({Q_{12}}) + {{x - {x_1}} \over {{x_2} - {x_1}}}f\left({{Q_{22}}} \right)

In the implementation process, Q₁₁, Q₁₂, Q₂₁, Q₂₂ is 4 adjacent pixels, and the p point falls on one of the above four segments. Let the upper left corner in the area be the area origin, and the horizontal distance of the original location of the area to which the target pixel belongs is Δ_col, then the colour calculation formulas of R₁ and R₂ in the x direction are as follows (28) δ(R1)=(Color(Q21)−Color(Q11))⋅Δcol+Color(Q11)⋅256 \delta \left({{R_1}} \right) = \left({Color({Q_{21}}) - Color\left({{Q_{11}}} \right)} \right) \cdot {\Delta _{col}} + Color({Q_{11}}) \cdot 256 (29) δ(R2)=(Color(Q22)−Color(Q12))⋅Δcol+Color(Q12)⋅256 \delta \left({{R_2}} \right) = \left({Color({Q_{22}}) - Color\left({{Q_{12}}} \right)} \right) \cdot {\Delta _{col}} + Color({Q_{12}}) \cdot 256 where Color(X) represents the colour value of point X. The specific calculation adopts 24-bit true colour format. The interpolation calculation formula for the y direction is as follows (30) f(R1)≈y2−yy2−y1f(R1)+y−y1y2−y1f(R2) f\left({{R_1}} \right) \approx {{{y_2} - y} \over {{y_2} - {y_1}}}f({R_1}) + {{y - {y_1}} \over {{y_2} - {y_1}}}f\left({{R_2}} \right)

If the ink image consists of pixels, there will be MN vectors c(x, y), though the random noise in the ink image has the same properties in the colour channels of the ink image, the effect on each colour channel is different. The median filter completes the filtering by calculating the aggregate distance from the filter window point to other points, and selecting the minimum distance pixel to replace the current pixel of the colour ink image. Let S_xy represent the pixel coordinates of the square neighbourhood with the centre located in (x, y), and perform median filtering on the colour ink image, the calculation formula is as follows (31) f(x,y)=middle(Sxy) f(x,y) = middle({S_{xy}}) (32) Sxy={im(x,y)|x∈[x−w,x+w];y∈[y−w,y+w]} {S_{xy}} = \left\{{im(x,y)\left| {x \in \left[ {x - w,x + w} \right]} \right.;y \in \left[ {y - w,y + w} \right]} \right\} f (x, y) represents the median filtered (x, y) value. S_xy represents the rectangular area of 2w + 1 and im(x, y) the pixel value in the rectangular area. Let the vertical distance from the target pixel to the origin of the area to be Δ_row and the origin of the area to be the same as the upper left corner of the area is as shown in Eq. (33). (33) Color(P)=(δ(R1)⋅256+δ(R2)−δ(R1))⋅Δrow) Color(P) = (\delta ({R_1}) \cdot 256 + \delta ({R_2}) - \delta ({R_1})) \cdot {\Delta _{row}})

Through linear interpolation in the x and y directions, the pixels of the source point are finally transported to the target position to achieve the scaling of the original ink image. The effect of the ink image calculated by the bilinear interpolation algorithm is relatively good, and there will be no obvious distortion and discontinuity, which is a good anti-aliasing technology. The size of the ink stroke is adjusted by the bilinear interpolation algorithm to fit the width of the spline model.