A study of utilizing computer-generated art methods to enhance creative expression in art education

Generative art is broadly considered to be the introduction of systemic factors into the practice of art making by the artist relinquishing the right to control the outcome of the work. With the passage of time, artworks can reflect the autonomy of the system, which is reflected in the birth and flux of programs, behaviors, and methods. In generative art, the most important thing is the construction of behavioral systems. Generative art is inclusive and serves as the foundation for the theoretical advancement of the fusion of technology and art. With the development of contemporary art forms, mechanical devices, microorganisms, ecosystems, natural language commands, computer algorithms, program coding, body movements, intelligent materials, robotic systems, computer data mapping, and other autonomous systems with valid rules have been introduced into the process of art creation, consisting of a continuous sequence of time from extreme order to extreme disorder. These can also be recognized as the systematic construction of “generative art” [25]. This system requires the artist not to presuppose a given result, but to develop art and extend the infinite possibilities of the creative medium in the construction of the system, placing the construction of the artistic act and process and the presentation of the result of the work of art on the same scale of importance. In addition to the influence of environmental art, ephemeral art, body art extreme art, conceptual art and light art on the creative trend of contemporary art, the systematic thinking of generative art and “relational design”, as a kind of creative thinking, is associated with almost any kind of creative ideas, media, expression and results of contemporary art, but has been neglected by theoretical researchers of art and design. However, theoretical researchers of art and design have neglected it. It was not until the advent of the artificial intelligence era that artists gradually realized the correlation between the autonomous system in the artwork and the creative subject.

Generative art in the narrow sense is computer-generated art, i.e. algorithmic art. The computer is granted autonomy by the artist by applying the concept of design, and by creating a programming language, the artist establishes specific rules to provide the computer with a great deal of autonomy. The computer creates programs automatically that create artworks related to vision, resulting in unique visual forms that cannot be replicated. Its development is highly relevant to both the visual arts and computer graphics. In fact, computer-generated images are more precise and abstract than traditional easel paintings. At the heart of the creation of computer-generated art lies the process of the artist designing the algorithm. The convergence of human and computer collaboration makes computer-generated art more than just a tool for interdisciplinary creativity in the fields of art and technology. The artistic result it presents also belongs to the research category of “aesthetics and computation”, which is an important issue in the study of aesthetics in the era of artificial intelligence. In the interactive video installation, each video experience of the audience is a unique real-time rendering, and the result is unexpected and difficult to copy and describe. The themes and aesthetics expressed are related to the connection between humans and nature, or the beauty and power of nature, whether known or unknown, or the cognition and actions triggered by human experience and behavior. Real-time, interactivity, mutability, and narrative are the most important characteristics of interactive video installations. As the experience moves, images, refracted light, sound, and even smells in the space echo their behavior, creating a fleeting and unrepeatable effect. The result is a fleeting and unrepeatable effect that creates a dazzling and profound impression.

Interactive images are forms of expression that explore connections between phenomena in different fields of perception. In a situation different from real life, the image, with the help of the artist’s way of perceiving the world, puts the viewer in a changing world full of various colors of light and shadow, silence and sound, call and response [26]. Computer-generated art introduces new perspectives on aesthetic interventions in perception through the construction of situations, time-spaces, and environments for the creation of video art. The transition between perception is gradually opened up, and can be “perceived” seems to be the concept of “unity of heaven and man” in the Eastern philosophy to achieve a modern scientific quantitative fit, and become the fusion of art and science in the creation of the way. The artist chooses the appropriate programming language and rules to correlate and transform various factors.

The creation of an interactive video installation experience is divided into five stages: connection, integration, interaction, transformation, and emergence. First, the audience must enter the space that emphasizes the here and now, connecting with the work and integrating it into the narrative of the image as the protagonist and the first viewer. They interact with the work, the artist, and the audience, resulting in morphological and narrative changes in the work. The relationship, thinking and experience of the image will also be updated in real time, resulting in the necessity to connect and become fully integrated (not just viewed from a distance), interacting with the system and others. This leads to a transformation of work and consciousness, resulting in the emergence of entirely new images, relationships, thoughts, and experiences.

The GAN network model draws on the idea of a two-player zero-sum game to estimate the generative model through the adversarial process, i.e., assuming that both parties simultaneously adopt the optimal solution and use it during the game, so that the outcome of the game reaches the state of Nash equilibrium. In this algorithm, generative model G and discriminative model D are trained together. Where G is responsible for learning the sample data distribution and D estimates the probability that the samples come from the generator G or the training data. The generative model needs to minimize the probability that the discriminative model makes a correct judgment on the data, while the discriminative model has to constantly identify the fake data generated by the generative model, and the two models fight against each other to improve each other, so that the data generated by the generative model has very little difference with the real data in terms of the data distribution, and the discriminative model is unable to determine whether the data is from the real dataset or is generated by the generator G generated.

Generative Adversarial Networks proposed by Good Fellow, the discriminator network uses conventional training methods, while the training data consists of a small dataset of real samples and random noise z, the real samples are labeled as 1, and the fake samples generated after the input of noise to the generator network are labeled as 0. The new dataset is denoted by G(z). The discriminative model D then has two inputs, x representing the real data and the data generated by the generative model G. The distribution of the real data is denoted by P_data(x), and the discriminative model makes a judgment on the data and outputs a scalar value.Then do back propagation to update the parameters of the generator network, the parameters of the discriminator network remains unchanged, alternating training, after many iterations, the data distribution of the fake samples is almost the same compared to the real data, that is, the generator network generated by the fake data discriminator has not been able to correctly out of the authenticity of the data, generating the adversarial network training process is shown in Figure 1.

Figure 1.

The training process of Generative Adversarial Nets

When the input is a true sample, the output of the discriminator network is close to 1. When the input is a false sample, the output of the discriminator network should be close to 0. Therefore, the loss function (or cost function) of the discriminator network D is defined as: (1) L(D)=−Ex−pdata(x)[ log(D(x)) ]−Ez−pz(z)[ log(1−D(G(z))) ] where p_data(x) denotes the distribution of the true sample data x, p_z(x) is the distribution of the input noise z, D(x) is the probability of x coming from the true sample data set, and G(z) denotes the sample data generated when the input to the generator network is z. Similarly, the generative model G is trained with the goal of fooling the discriminator network as much as possible so that the discriminator thinks that the generated fake samples G(z) are from the real data set [27]. Therefore, the loss function of generator network G is: (2) LL(G)=Ez−pz(z)[ log(1−D(G(z))) ]

In order for the discriminator to make as many errors as possible, L^(G) should be minimized, and as it can be seen from Eqs. (1)(2) that L^(G) is closely related to L^(D), the loss function of the whole network can be written as: (3) V(D,G)=−L(D)=Ex−pdata(x)[ log(D(x)) ]+Ez−pz(z)[ log(1−D(G(z))) ]

That is, if the training of a generative adversarial network is a very small and very large game problem, then its optimization process requires that the loss function V(D,G) be maximized in the inner layer and minimized in the outer layer, i.e., minG maxDV(D,G) .

Assuming that x = G(z) is invertible, an equivalent transformation on a continuous space has: (4) minG maxDV(D,G)=Ex−pdata(x)[ log(D(x)) ]+Ez−pz(z)[ log(1−D(G(z))) ]=∫xpdata(x)log(D(x))dx+∫xpz(z)log(1−D(G(z)))dz=∫xpdata(x)log(D(x))dx+∫xpg(x)log(1−D(x))dx=∫xpdata(x)log(D(x))dx+pg(x)log(1−D(x))dx

With the generative model given, it can be shown that the optimal discriminator that can be obtained at this point is: (5) maxDV(D,G)=∫xpdata(x)log(D(x))dx+pg(x)log(1−D(x))dx (6) ∂∂D(x)( pdata(x)log(D(x))+pg(x)log(1−D(x))=0 (7) pdata(x)D(x)−pg(x)1−D(x)=0 (8) DG*(x)=pdata(x)pdata(x)+pg(x)

Johnson et al. draw on the design ideas of DCGAN and VGG network model to propose a real-time fast image style migration method. The real-time image style migration algorithm flow is shown in Figure 2, and the model contains two networks, the image transformation network and the loss network. The input image x is processed in the training process of the image transformation network and outputs the generated image data y^ . The loss network, on the other hand, extracts features and calculates losses for the generated image y^ , the style image y_s and the content image y_c (i.e., the input image x), respectively. f_W As an image transformation network, it is the structure of a deep residual network, replacing the regular pooling process with stepwise convolution as well as fractional stepwise convolution, with convolution kernels of 3×3 for all convolutional layers except for the first and the last convolutional layer, which uses a convolutional kernel of 9×9. BN and ReLU are added after the non-residual convolutional layers except for the output layer, where the output pixel values are bounded by the tanh function with an output range of [0,255].

Figure 2.

Real-time style transfer algorithm process

The VGG network is trained on the Image Net dataset, and the resulting pre-trained model is constituted as a loss network. Feature reconstruction loss L_feature and style reconstruction loss L_style are added with appropriate weight parameters λ₁ and λ₁ to form the visual loss function: (9) L=Ex,yc,yc[ λ1Lfeature(fW(x),yc)+λ2Lstyle(fW(x),ys) ]

The characteristic reconstruction loss is: (10) Lfeaturej(y^,y)=1CjHjWj‖ ϕj(y^)−ϕj(y) ‖22

Where, C_j, H_j, and W_j denote the number of channels, height, and width of the output feature maps of the jth layer convolutional network, respectively, and the feature maps output from the jth layer convolution are denoted by (y) in the loss network. The F–parameter of the difference between the output image and the Gram matrix of the target image constitutes the style reconstruction loss after the convolution of the jth layer as follows: (11) Lstylej(y^,y)=‖ Gj(y^)−Gj(y) ‖F2 where the Gram matrix G_j(x) is a matrix of size C_j×C_j with matrix elements: (12) Gj(x)c,c′=1CjHjWj∑h=1Hj∑ω=1Wjϕj(x)h,ω,cϕ(x)h,ω,c′

The style reconstruction loss is the sum of the style reconstruction losses after the convolution of the 2nd, 4th, 7th and 10th layers. Compared to the image style migration algorithm proposed by Gatys et al. This real-time image style migration algorithm and the method of Gatys et al. are both built on the basis of convolutional neural networks, although the former image generation effect is slightly worse, after the training of the image transformation network is completed, it is only necessary to do the forward computation of the image transformation network only once for each input image, instead of doing heavy neural network training, the training speed is increased by more than 200 times.

Since GAN was proposed, there have been many papers investigating the loss functions of GAN such as wgan, hinge loss. Among them hinge loss is proved to be the most stable in most experiments and performs faster in unconditional image generation. In this paper, we use the hinge version of adversarial loss to iteratively train the discriminator D and generator G: (13) LD=−Ex~Pd[ min(0,−1+D(x)) ]−Ez~N[ min(0,−1−D(G(z))) ] (14) LG=−Ez~N[ D(G(z)) ]

Unlike existing GAN architectures, the model of Draw GAN consists of a generator with three outputs that are connected to three different discriminators. Although the model architecture in this paper is designed to resemble an artist’s drawing process, it also conforms to the GAN training mechanism. Unlike traditional GAN generators that typically produce a single view image y, Draw GAN’s generator takes a noise vector z~N as input and produces an output of an image set {y_sketch, y_gray, y_color}. Similarly, in order to train DrawGAN, in this paper, the line art, gray scale and image itself of a real image are first extracted in the preprocessing stage as a real image set {x_sketch, x_gray, x_color}. Corresponding discriminators D_sketch, D_gray, and D_color are constructed to differentiate the real/fake views, respectively. Therefore, the formula can be rewritten as: (15) LDi=−E[ min(0,−1+Di(xi)) ]−E [ min( 0,−1−D(yi) ] where i ∈ {sketch, gray, color}, {y_sketch, y_gray, y_color} = G(z)

This section adds an auto-encoder to the discriminator to achieve self-supervised learning by reconstructing the input. A self-encoder is a specific type of feed-forward neural network in which the inputs are the same as the outputs.

The structure of a self-encoder, shown in Figure 3, compresses the input into abstract features and then reconstructs the output from that representation. Abstract features are a kind of “compressed” information of the input, which is also called potential space representation.Its main purpose is to convert input x into abstract features and then x into output x˜ . By comparing x and x˜ so that they are infinitely close to one another.

Figure 3.

The structure of the self-code device

By using the reconstruction loss of the self-encoder to provide strong regularization for discriminator D, it can be ensured that D can extract a more comprehensive representation from the input, which does not incur much extra computation. The formula for its reconstruction is as follows: (16) Lrecons=Ef~Dencode(x),x~Ireal[ ‖ g(f)−x ‖ ]

Where ‖·‖ indicates that the reconstruction function can be selected ℓ₁, ℓ₂, LPIPS and other functions, f is the intermediate feature mapping of D, and the function g indicates the decoder, which is responsible for reducing the intermediate feature mapping f into a network of images.

In order to model can be expanded to more less datasets, this paper adds the program of micro-able data enhancement, training micro-able data enhancement uniform choice, first random color adjustment, then random panning, and finally random cropping. Then the loss function of Draw GAN can eventually be rewritten as: (17) LD=−E(Dsketch(ysketch))−E(Dgray(ygray))−E(Dcolor(ycolor)) (18) LD=LDsketch+LDgray+LDcolor (19) LDi=−E[ min(0,−1+Di(T(xi))) ]−E [ min( 0,−1−D(T(yi)) ] +E[ ‖ g(f)−T(xi) ‖ ] where T denotes microdata enhancement.

Based on the introduction of the above formulas, this paper designs a generative adversarial network training architecture with three outputs of multi-view generators, and a combination of multi-discriminators, and the training framework of DrawGAN is shown in Fig. 4.

Figure 4.

DrawGAN’s training framework

This subsection first introduces the multiview generator of Draw GAN, which will first generate the line drawing and grayscale of the image, and then finally the color map.The generator network architecture of Draw GAN is shown in Fig. 5, which starts by converting the input noise vector z into a feature space of 4×4 resolution by transposing convolution (4×4) and convolution (3×3). After that the resolution of the feature space is increased each time by a multiple of 2. Each increase in feature space resolution consists of an adjacent upsampling and two 3×3 convolutions, which follows the design scheme of most GANs. Finally a mapping function r is defined, which in this paper is simply r modeled as a convolution (1×1) by which the feature space is projected onto the image space. It generates outputs at different stages of the generator, where the outputs correspond to line drawings, gray and color images (color images are always the outputs obtained through the mapping function in the last feature space).

Figure 5.

The Draw GAN generator network architecture

In order to synthesize higher resolution images, generator G inevitably needs to become deeper with more transformation layers to match the upsampling. Due to the increase in the number of model parameters and the weaker gradient flow through G, the longer the training time of the GAN. In order to better train deep models, the structure of residual blocks is proposed, which uses jump connections to strengthen the gradient signal between layers.

The idea of jump connectivity is combined with the channel attention mechanism by using a jumping channel attention module (SSE).The structure of the SSE module is shown in Fig. 6, which first down-samples the underlying feature mapping x_low into a feature map of spatial dimension of size 4×4 through an 4×4 adaptive average pooling layer, followed by a transformation layer that further down-samples it to 1×1. Then a transformation module is used to adjust the number of feature channels to the same channel size as x_high. Finally, after a gating operation via a sigmoid function, the output is multiplied by x_high along the channel dimensions to obtain x˜high with the same shape as x_high.

Figure 6.

The structure of the SSE module

The generated image type classification discrimination results are shown in Figure 7. In order to present the results visually, the horizontal axis represents the degree of certainty of each type based on contour and the vertical axis represents the degree of certainty of each type based on texture. The analysis reveals that the types with a degree of certainty higher than 50% are:The labels “C”, “F”, “I”, “K”, “M”, “R”, “U”, “K” and “R” ranked first and second with 61% and 63% certainty, respectively, and all met the needs of this study in terms of ambiguity in generating images.The “U” label of the image was ranked third with a certainty of 59%, and its “transparent” characteristics are in line with the ethereal and hazy artistic style presented by the base image, so the algorithm validation was successfully passed.

Figure 7.

Generate the classification of image types

The experiment invites 30 students from a school to participate in an image scoring activity generated using the methods in this paper, and the ideas and opinions of these 30 students are summarized and analyzed using a questionnaire.

Figure 8.

Degree of preference

Before conducting the experiment, the students’ creative self-efficacy, creative tendencies and their performance in various dimensions were tested, and independent samples t-tests were conducted on the members of the experimental and control groups using the independent samples t-test method, and the differences between the experimental group and the control group in the different dimensions in the pre-test stage are shown in Table 3 (* indicates p<0.05, ** indicates p<0.01, and *** indicates p<0.001). As can be seen from the table, there is no significant difference between the experimental group and the control group on the various dimensions, and the two groups of students are basically at the same level and belong to a homogeneous group. Therefore, they can be compared as equivalent groups.

In order to investigate the effect of the method proposed in this paper on students’ creative self-efficacy and creative tendencies, the differences in the scores of the experimental and control groups on the dimensions at the post-intervention stage are compared. The differences between the experimental and control groups on the different dimensions at the posttest stage are shown in Table 4 (* indicates p<0.05, ** indicates p<0.01, and *** indicates p<0.001).As can be seen from the table, at the end of the experiment, there was a significant difference in the scores of all dimensions between the two groups, with the experimental group scoring higher than the control group. Specifically, in terms of imagination, the experimental group (M=14.93, SD=2.43) scored significantly higher than the control group (M=13.26, SD=2.13), t=-3.441, p=0.005<0.01.In terms of creative strategies, the experimental group (M=3.04, SD=0.95) was also significantly larger than the control group (M=2.64, SD=0.51), t=-2.839, p=0.006<0.01.On the dimension of creative finished product, the experimental group (M=3.69, SD=0.52) scored significantly higher than the control group (M=2.76, SD=0.59) as well, t=-2.524, p=0.017<0.05. Summarizing the statistical results mentioned above, at the end of the experiment, the scores of the vast majority of dimensions of the experimental group were significantly higher than those of the control group, which indicates that teaching with the method of this paper in art education has a positive impact on improving students’ creative tendencies.

The differences between the experimental groups in the pre-test and post-test phases on the different dimensions are shown in Table 5 (* indicates p<0.05, ** indicates p<0.01, and *** indicates p<0.001). The results showed a significant improvement in the scores of the vast majority of dimensions in the experimental group after applying the method proposed in this paper to art education. Specifically: on the creative strategy dimension, there was a significant difference (t=-3.244, p=0.008<0.01) between the pre-test (M=2.02, SD=0.27) and post-test (M=3.28, SD=0.77) in the experimental group, with the post-test scores being significantly higher than the pre-test scores. On the dimension of creative finished product, there was a significant difference between the pretest (M=2.25, SD=0.75) and posttest (M=2.83, SD=0.27) of the experimental group (t=-5.549, p=0.000<0.001), with the posttest scores being significantly higher than the pretest scores. To summarize, the scores of the experimental group significantly improved in the vast majority of dimensions. This indicates that art education using the method presented in this paper is effective in improving students’ creative abilities.

The differences between the pre-test and post-test stage control group on the different dimensions are shown in Table 6 (* indicates p<0.05, ** indicates p<0.01, and *** indicates p<0.001). As can be seen from the table, within the control group, the difference between the scores of the pre-test and post-test on each dimension was not significant. This indicates that in the absence of intervention, there was no significant change in the scores of the control group between the pre and post-tests on each dimension.