Research on the Expanded Night Road Condition Dataset Based on the Improved CycleGAN

In generative networks, the residual structural unit normalization [5] used is an algorithm based on adaptive normalization that speeds up the stylization of an image while maintaining its style. Here, we first input the content and style data at the same time, then go through the four residual modules, and finally the style transfer is completed by the up-sampling and convolution operation.

Neural networks for different tasks often choose different normalization functions, and different normalization functions will have different effects on the final results. Compared with the original GAN model, the CycleGAN model replaces batch normalization with instance normalization in order to make the model more stable.

CycleGAN uses instance normalization to perform separate normalization processing on each image, making the content information between each image independent of each other, avoiding the mutual influence between images and resulting in blurred generated images. Some progress has been made using instance normalization, but this method only works better in image style conversion when there are small differences in the shape and texture of the image domain. In aspects such as face style conversion, which have large differences, the effect is not very ideal. Instance normalization operates on each image and normalizes the H and W dimensions on a single channel, so in IN, different channels of different features are irrelevant, which allows the content feature information to be transferred to the style feature When applied to information, good results will always be achieved, but if it is only applied to a single channel, it will interfere with the original semantic information. Layer normalization acts on all channels. Layer normalization normalizes the C, H, and W dimensions of each image. It considers more global information, so some detailed information is often ignored. During training, the parameters of adaptive layer instance normalization (AdaLIN) [11], can be learned from the data set by adaptively selecting the proportion between instance and layer normalization. Adaptive layer instance normalization combines the two, offsets their shortcomings, and combines the advantages of both. The specific formula is as follows: (1) ∂^I=∂−μIσI2+ε,∂^L=∂−μIσI2+ε {\hat \partial_I} = {{\partial - {\mu_I}} \over {\sqrt {\sigma_I^2 + \varepsilon}}},\,{\hat \partial_L} = {{\partial - {\mu_I}} \over {\sqrt {\sigma_I^2 + \varepsilon}}} (2) AdaLIN(∂,γ,β)=γ.(p.∂^I+(1−p).∂^L)+β A{\rm{da}}LIN\left({\partial,\gamma,\beta} \right) = \gamma.(p.{\hat \partial_I} + (1 - p).{\hat \partial_L}) + \beta

The adaptive layer instance normalization method is introduced into the decoder of the generator network, and combined with the self-attention mechanism, important local feature information and global feature information are automatically learned during network training.

Figure 5.

The migrating image is the result of the generative network, which takes style and content elements as inputs.

Convolutional neural network [6] is still used in feature extraction of the whole image, and then recognition is performed based on the extracted features. Nevertheless, this approach frequently overlooks the image features, leading to issues like detail loss and image blurring. Therefore, we use a multi-scale discriminative network, which is an improved chunked discriminative network.

Figure 6.

The PatchGAN discriminator is essentially a multi-layer convolutional network. The input image of the network undergoes multi-layer feature extraction and finally outputs a 30*30 feature map. The network also divides the input image into 30*30 patches. The pixels in the feature map match the input image patch, and the value in the feature map is between 0 and 1, which is employed to convey how close the patch component image is to the original image.

Finally, to show how similar the complete image is to the original image, the scores of each patch are put together and averaged. The PatchGAN discriminator can identify patches that are significantly different from other patches in image information and give the corresponding patch a lower discriminant score. This way, the local information of the generated image can be consistent with the overall information, which significantly lowers the expense of producing the picture mistake. In addition, because the local image and the overall image are closely related, the patchGAN network based on local image area discrimination plays a positive role in the fusion of the overall information and local information of the image.

The discriminative network has one fully connected layer and four convolutional layers making up its network structure.

Adversarial loss [7] is used to bring the generated image closer to the actual target style image. Equation displays the adversarial loss computation formula. (3) LGANy1=Ec1−p(c1),s2−q(s2)[log(1−D1(G1(c1,s2)))]+Ey1−p(Y),s2−q(s2)[log D1(y1)] \matrix{{L_{GAN}^{{y_1}}} \hfill & {= {E_{{c_1} - p\left({{c_1}} \right),{s_2} - q\left({{s_2}} \right)}}[\log (1 - {D_1}({G_1}({c_1},{s_2})))]} \hfill \cr {} \hfill & {+ {E_{{y_1} - p\left(Y \right),{s_2} - q\left({{s_2}} \right)}}[\log {D_1}({y_1})]} \hfill \cr} (4) LGANx1=Ec2−p(c2),s1−q(s1)[log (1−D2(G2(c2,s1)))]+Ex1−p(X),s2−q(s2)[log D2(x1)] \matrix{{L_{GAN}^{{x_1}}} \hfill & {= {E_{{c_2} - p\left({{c_2}} \right),{s_1} - q\left({{s_1}} \right)}}[\log (1 - {D_2}({G_2}({c_2},{s_1})))]} \hfill \cr {} \hfill & {+ {E_{{x_1} - p\left(X \right),{s_2} - q\left({{s_2}} \right)}}[\log {D_2}({x_1})]} \hfill \cr}

G₁ (c₁, s₂) denotes the output migration image, and D₁(G₁ (c₁, s₂)), D₁(y₁) are the discrimination results, respectively.

In the article, the generative network is fed both the style and content elements that were taken out of the style and content encoders in order to improve the capability of generative network in generative adversarial network.

The model is characterized by clearer and richer images clearly state the units for each quantity that you use in an equation.

Because a content encoder is included, the content coding loss is added to the loss function. The article proposes a method for style transfer using mismatched data and content feature extraction of the image using the content encoder, which ensures the invariance of the image during the style transformation. The style-shifted image [8] is re-inputted into the content encoder Ec, and as much as feasible, the content information of the picture is passed through the content information of the image that is output by the content encoder c so that the content information obtained has the same degree as the actual image described in order to train the content encoder.

The style encoding loss is added to the loss function since the style encoder is now included. In order to ensure that the style converted image maintains the same style as the target image, the modified image is once again entered into the style encoder Es, and the extracted image style information must be as similar as possible to the style information outputted via the style encoder, so as to achieve the training of the style encoder.

CycleGAN network model, both to stylize the image and to recover the transformed image again, so the cyclic consistency loss of the image needs to be considered [9].

The training process of the network optimizes the parameters using Adam's algorithm [10], this uses adaptive low-order moment estimation as the basis for a one-time gradient optimization of any objective function.

It mainly contains the following features: 1)

Straightforward realization

We will train the content encoder, style encoder, generator, and input the content and style images to the appropriate content and style encoders for feature extraction during the project's testing phase, and then input the extracted content from the encoder to the generator to finally produce the style migration image on the generator.

Below are some loss function images from the training:

Figure 7.

Specific values for some of the losses are listed in Table I:

During the actual training process, the training results can be judged according to the loss value. The smaller the value, the more successful the training is. Generally, the training results improve with a lesser decrease of D. As can be seen from Figure 7, as training continues, each loss value has a decreasing trend. It can also be observed during the training process that the effect of style transfer gradually becomes stronger as the epoch increases.

There are also some demonstrations of images during training, as in Figure 8:

Figure 8.

It can be seen that at the beginning of the training, the effect is not very obvious. As the number of iterations and training increases, the style migration can slowly be seen, and eventually the transformation is completely clear.

The topic of autonomous driving has advanced quickly in recent years, and the quality and quantity of datasets are crucial for the research in the domain of self-driving cars, and excellent datasets can help the autonomous.

Diving system better adapt to different lighting and environmental conditions, to enhance the self-driving system's functionality and safety at night and in low light conditions. At night or in low-light conditions, objects in images are often more difficult to identify and locate than during the day, which can lead to degraded performance of autonomous driving systems, increasing the risk of traffic accidents. Therefore, the dataset can be enriched and enhanced by using the method of daytime and nighttime road map style transfer, to enhance the capacity for generalization and the training effect of the autonomous driving system.

In this experiment, 1000 actual traffic maps were selected as the training set using the BDD100K dataset, including 500 road maps during the day and 500 road maps during the night period. 400 actual traffic maps were used as training sets, including 200 during the day and 200 at night.

The conversion effect is shown in Figure 9:

Figure 9.