1. bookAHEAD OF PRINT
Detalles de la revista
License
Formato
Revista
eISSN
2444-8656
Primera edición
01 Jan 2016
Calendario de la edición
2 veces al año
Idiomas
Inglés
Acceso abierto

Computer Vision Communication Technology in Mathematical Modeling

Publicado en línea: 15 Jul 2022
Volumen & Edición: AHEAD OF PRINT
Páginas: -
Recibido: 09 Feb 2022
Aceptado: 28 Mar 2022
Detalles de la revista
License
Formato
Revista
eISSN
2444-8656
Primera edición
01 Jan 2016
Calendario de la edición
2 veces al año
Idiomas
Inglés
Introduction

Helping the visually impaired restore or improve vision has been a dream of humankind for many years. It is also the research direction that scientific and technological workers have been committed to for a long time. Among them, artificial vision compensation technology has made great progress in recent years [1]. Visual impairment is generally caused by damage to certain parts of the visual pathway, such as the retina. The basic method of artificial vision compensation is to implant a microchip in the optic nerve pathway to generate a certain electrical signal to stimulate the optic nerve cell. This causes its excitement to generate neural action potentials transmitted to the primary visual cortex and higher visual center to induce human visual perception. We propose a pixelated imaging model based on the generation of salient local features. The article designs a simulation evaluation experiment based on the subjective evaluation scoring method to analyze the performance of this model. The article hopes to provide a reference for follow-up research in this field.

Pixelated imaging model based on local saliency features

The research results show that the original image has some local structural features that are of interest to visual institutions. They do not need to go through high-level visual behaviors such as classification, extraction, and recognition. These local areas with obvious attractive properties are “significant” areas. Its “inspiration” to the eyes gives people the feeling that the area is rich in visual information [2]. On the one hand, although the “salience area” has different structural forms, visual perception can all be noticed. On the other hand, there is a “side inhibition” effect between visual perception units, and there is a certain spatial distribution “receptive field” for the excitation signal. Based on this, a pixelated imaging model frame based on local saliency features is shown in Figure 1.

Figure 1

Pixelated imaging model framework based on locally saliency features

The processing flow of this model on the original input image includes:

(1) Use several saliency feature detectors to extract the original image's various local saliency structure maps. (2) Implement “competition” for each mapping to reproduce the enhancement and inhibition of the visual receptive field, and then “fusion” all feature maps to obtain the final saliency map. (3) We implement multi-resolution pixelization of the original image and sub-sampling to hundreds of pixels based on the final feature mapping. Areas with stronger salient features give finer resolution, and vice versa. Subjectively, strong saliency areas rich in visual information are given priority to the subjects to achieve the effect of delivering as much visual information as possible.

Local saliency feature detector

First, the contrast saliency map needs to be obtained. The contrast of a local area relative to the surrounding area can be quantitatively described by Michaelson contrast. The Michaelson contrast at pixel position (x, y) is C(x,y)=|LmLM|Lm+LM {\rm{C}}\left({x,y} \right) = {{\left| {{L_m} - {L_M}} \right|} \over {{L_m} + {L_M}}} Lm is the average gray value of (x, y) in a 7×7 area. LM is the average gray value of the whole image.

Secondly, it is necessary to obtain the edge density saliency map. The edge density describes the strength of the effective edge in the unit area of the original image. First, we need to select a suitable edge extraction method to obtain meaningful edges and then calculate the edge strength in each region [3]. An operator that can effectively detect important edges, suppress noise, and accurately locate edges is the Canny operator. We give a Gaussian filter G and use the following formula to estimate the normal unit vector of the local edge n=(GI)(GI) n = {{\nabla \left({G \otimes I} \right)} \over {\left\| {\nabla \left({G \otimes I} \right)} \right\|}}

The location of the edge satisfies 2n2GI=0 {{{\partial ^2}} \over {\partial {n^2}}}G \otimes I = 0

The strength of the edge is equal to the denominator of equation (2). We threshold the detection results to remove weak responses and perform edge synthesis to obtain meaningful edges.

Then it is necessary to obtain the directional difference saliency mapping. The so-called directional difference can be illustrated in Figure 2(a). In this original image with texture attributes, most of the primitives are close to the horizontal direction. In the experiment, 4 primitives are different, so the area centered on these 4 primitives shows a difference in directionality with the surroundings [4]. When viewing this image, the line of sight will be attracted by 4 primitives different from the surroundings. Therefore, these areas with different directionality from the surrounding area belong to the salient feature area of interest. The directional difference detection operator we selected uses a 2-dimensional Gabor filter Ga(x,y)=exp{x2+y22σ2}cos(2πf(xcosθ+ysinθ)) Ga\left({x,y} \right) = \exp \left\{{{{{x^2} + {y^2}} \over {2{\sigma ^2}}}} \right\}\cos \,\left({2\pi f\,\left({x\cos \,\theta + y\,sin\,\theta} \right)} \right)

Figure 2

Example of directivity difference and Gabor filter detection result

We use a Gabor filter with an appropriate center frequency. The direction angles are 0°, 45°, 90°, and 135°. The test is performed in Figure 2(a), and the results are as shown in 2(b)–(e). The response effect in the 4 local areas of interest is always different from the surroundings. Combining the four response results can get the directional difference significance map is shown in Figure 2(f). The highlighting in the 4 local areas of interest indicates a significant difference in directionality with the surrounding area.

Finally, the symmetry saliency mapping needs to be obtained. Based on the literature, define the contribution of pixel positions (x + u, y + v) and (xu, yv) to the symmetry of the center (x, y) as Sx,y(u,v)=1|pn(x+u,y+v)+pn(x,u,yv)pn(x+u,y+v)+pn(xu,yv),q(u,v)q(u,v)| {S_{x,\,y}}\left({u,v} \right) = 1 - \left| {{{{p_n}\left({x + u,\,y + v} \right) + {p_n}\left({x,u,\,y - v} \right)} \over {\left\| {{p_n}\left({x + u,\,y + v} \right) + {p_n}\left({x - u,\,y - v} \right)} \right\|}},{{q\left({u,\,v} \right)} \over {\left\| {q\left({u,\,v} \right)} \right\|}}} \right| pn (m, n) represents the normalized gray gradient vector at the original image (m, n). q(u, v) represents the direction vector connecting the two points (x + u, y + v) and (xu, yv). The final symmetry measure at the point (x, y) is S(x,y)=uvp(x+u,y+v)p(xu,yv)Sx,y(u,v) S\left({x,y} \right) = \sum\limits_u {\sum\limits_v {\left\| {p\left({x + u,\,y + v} \right)} \right\|\,\left\| {p\left({x - u,\,y - v} \right)} \right\|} \,{S_{x,\,y}}\left({u,\,v} \right)}

The saliency mapping of the original image's contrast, edge density, directional difference, and symmetry feature can be obtained by the effects of the above-mentioned salient feature detectors on the original image.

Competition and integration of feature mapping

When we use the detectors, as mentioned earlier, to act on the original image, there will always be a response to the non-uniform grayscale area in the image. Therefore, the entire saliency map will show strong and weak responses and scattered noise-like responses. The area of interest of the visual system is limited [5]. The imaging resolution of artificial vision systems is limited. Therefore, only certain areas where the saliency is concentrated and reach a certain intensity can be regarded as the truly significant feature areas. The competitive step is reserved for aggregated significant responses. This weakens other responses to get a limited salient area. If there is a characteristic excitation of unit intensity at (0, 0) and t0. The enhanced response felt at (x, y) and t can be expressed as r1(x,y,t)=c1exp(x2+y22σ12)(1+bea(tt0)) {r_1}\left({x,\,y,\,t} \right) = \,{c_1}\,\exp \left({- {{{x^2} + {y^2}} \over {2\sigma _1^2}}} \right)\left({1 + b{e^{- a\left({t - {t_0}} \right)}}} \right)

The inhibitory response felt at (x, y) and t can be expressed as r2(x,y,t)=c2exp(x2+y22σ22)(1+bea(tt0)) {r_2}\left({x,\,y,\,t} \right) = \, - {c_2}\,\exp \left({- {{{x^2} + {y^2}} \over {2\sigma _2^2}}} \right)\left({1 + b{e^{- a\left({t - {t_0}} \right)}}} \right)

Based on the above expression, a comprehensive reaction expression can be obtained R(x,y)=c1exp(x2+y22σ12)c2exp(x2+y22σ22) R\left({x,\,y} \right) = \,{c_1}\,\exp \left({- {{{x^2} + {y^2}} \over {2\sigma _1^2}}} \right) - {c_2}\,\exp \left({- {{{x^2} + {y^2}} \over {2\sigma _2^2}}} \right)

We use this template to convolve with the salient feature maps. This can approximately characterize the enhancement and suppression effects of the visual system on these features. The parameter c1, σ1, c2, σ2 determines the shape and size of the “competition” area. Laplacian of Gaussian template can be further used in the simulation of artificial vision compensation under subjective experiment LoG(x,y)=(1x2+y22σ2)exp(x2+y22σ2) LoG\left({x,\,y} \right) = \left({1 - {{{x^2} + {y^2}} \over {2{\sigma ^2}}}} \right)\,\exp \left({- {{{x^2} + {y^2}} \over {2{\sigma ^2}}}} \right)

Only one parameter of this template needs to be adjusted. This is simple and easy to perform imaging based on subjective simulation experiments. The significance of A at (x, y) is mapped to Fn(x, y) after the first detection function is completed and the above competition step is completed [6]. We want to merge them into a unified saliency map F(x, y). Before further visual, physiological and psychological evidence is generated, we will perform a linear weighted summation of each Fn(x, y). F(x,y)=n=1NwnFn(x,y) F\left({x,y} \right) = \sum\limits_{n = 1}^N {{w_n}{F_n}\left({x,y} \right)}

F(x, y) is the result of the first two steps of the imaging model. The result will be used to guide the process of multi-resolution pixelization, that is, sub-sampling, on the original image. The determination of the weight wn depends on the stimulation performance of various local saliency structures to the human eye. Appropriate weights need to be obtained through experimental feedback.

Pixelation

In the image segmentation algorithm, the splitting steps of the splitting and merging algorithm are easy to correspond to the imaging model with the phantom optical array as the target. The split and merge method needs to determine a uniformity criterion. If a square in the image does not meet this criterion, it is divided into 4 squares. The commonly used uniformity criterion is the gray-level variance threshold criterion. If the gray variance in a square is less than a certain threshold, it is considered uniform. The “uniformity” criterion is revised to the significant degree criterion [7]. The block whose significance is lower than a certain threshold is considered as no need to be divided, and the block is regarded as a phantom visual point. We use the average gray level in the square to represent the image point's imaging result; otherwise, we continue to divide it into four. Such a prominent square area will be presented to the observer at a higher resolution.

The subjective simulation evaluation experiment
Single feature experiment

The first implementation is a single feature experiment. Its purpose is to pass the test of each feature of contrast, edge density, directivity difference, and symmetry one by one. We examine their effects on visual perception functions. This provides a basis for selecting the weight coefficient of the feature fusion equation (11).

We divide the original image to be imaged into four scene category groups: the human face, ordinary object, indoor scene, and outdoor scene. Each group includes 12 256×256 grayscale images. Each image of each group extracts the salient features of contrast, edge density, directional difference, and symmetry according to the imaging model [8]. Then complete the “competition” of features. Since the object of investigation is a single feature, we control the result of each split to be as close to 400 small blocks (pixels) as possible. In this way, each original image gets 4 imaging results that reflect each salient feature. They become the subject of evaluation.

Nine university undergraduates and graduate students participated in the experiment as subjects. The subjects directly observed the imaging results with the naked eye from the computer display screen and compared them with the original image. The subjects were asked to score 1 to 5 points for each imaging result of each original image. High scores represent excellent imaging results [9]. The subjects were told that the excellent imaging results conveyed richer information to the observer. Whether it completely reconstructs the details of the original image is not important. In addition, the subjects were required to give a reasonable distribution of scores between 1 and 5 for each composition image result. This avoids uneven scores due to the different personal habits of the subjects.

Table 1 shows the score statistics of the feature imaging results in the four scene types. It is not difficult to see that different local features signify different local attributes of the original image. The visual system's sensitivity to it varies. The imaging results that highlight high-contrast areas have an advantage. This shows that the visual system prioritizes areas with strong contrast.

Single feature subjective scoring results statistics (mean ± standard deviation).

Feature Contrast Edge density Directional difference Symmetry
Human face 2.10±0.12 3.11±0.61 2.37±0.01 2.47±0.17
Typical objects 3.30±0.43 2.95±0.5 2.14±0.11 2.81±0.21
Indoor scene 2.88±0.17 2.69±0.17 2.24±0.08 0.41±0.17
Outdoor scene 3.07±0.17 2.24±0.11 2.19±0.08 2.88±0.19

The imaging results that highlight areas with high edge density followed closely behind. This verifies the important position of the edge in the understanding of visual information. The particular directional area and the symmetrical area are generally effective in the first three sets of scenes. Still, they are more prominent in outdoor scenes than the areas with higher edge density.

We take the imaging results reflecting the 4 local features as 4 totals. At the same time, we use the score of each imaging result as each overall sample to implement a one-way analysis of variance. The corresponding null hypothesis probability values are shown in Table 2. The null hypothesis in each set of scenarios at the significance level of 0.005 was rejected. We believe that different local salient features have obvious differences in the function of visual information transmission.

Single-feature imaging results, single-factor analysis of variance results.

Scene type
human face Typical objects Indoor scene Outdoor scene
Null hypothesis probability 3.8×10−3 3.3×10−4 4.2×10−4 1.2×10−4

The above experimental results show that different local salient features affect imaging results under different scene types. In terms of visual information transmission, they have their unique attributes and are restricted by the type of scene. This proves the necessity of using several local feature detectors to extract salient features separately [10]. The score ratio of each feature after pixelation provides a basis for selecting the weight coefficient of the feature fusion step.

Complete model experiment

In this experiment, each original image first uses the grayscale variance as the uniformity criterion to implement pixelation to generate a test image. This can be used to compare the imaging results produced by the complete processing flow of the model. Then, after the complete processing flow of the model, the feature fusion step is implemented to produce two imaging results. The mean value is taken when the saliency of each feature is merged. We take the weighting coefficient wk in equation (11) as 0.25. The weight of the other frame is generated based on the scoring result of the previous experiment. If the average score of the k feature is Sk, then the weighting coefficient of the k feature is wk=Sk1S1 {w_k} = {{{S_k}} \over {\sum\limits_1 {{S_1}}}}

In this experiment, the original images’ selection, quantity, grouping method, and presentation interface are the same as those in the previous experiment. The subjects did not change either. The subjects did not obtain any data about the results of the previous experiment, nor did they know how the image to be tested was generated [11]. The subjects were asked to score the 3 imaging results of each original image. It must contain a score of 0, a score of 1, and a score of 2. A high score means that the imaging results convey richer information. Because the total score of each set of scene types is a fixed 36 points. This examines the proportion of each imaging result in the total score. The scoring results of faces, familiar objects, indoor scenes, and outdoor scenes are shown in Figure 3 (a) ~ (d) in turn. Among them, white represents the percentage of scores that are pixelated according to the grayscale variance criterion. Black indicates the average pixelized score ratio of the feature. Gray is the pixelized score ratio after feature weighting according to the previous experimental result. After the pixelization process is performed on the feature map obtained by the saliency weighted summation, the score ratio of the image is dominant.

Figure 3

The percentage of full model experiment scores

The frequency of 2 points, 1 point, and 0 points obtained after the pixelization process is implemented according to the feature map obtained by the saliency weighted summation. The results are shown in Figure 4. The black bars, gray bars, and white bars represent the average times of 2, 1, and 0 points, respectively. It can be seen that the average number of 2 points is the most. This result indicates that the use of salient features as the criteria for the regional splitting process will provide the visual system with more subjectively richer visual information. The imaging results obtained using weighted feature summation to implement fusion are more successful than other imaging results. The above simulated subjective evaluation experiments have verified the rationality and effectiveness of the imaging model to a certain extent.

Figure 4

The average number of scores of imaging results weighted by significance

Conclusion

This article proposes a pixelated imaging model based on salient local features. We design a simulation evaluation experiment based on subjective evaluation scores to examine the performance of this model. This model can give the subjects preferentially presenting the characteristically significant areas. This allows the subjects to feel richer visual information subjectively. Therefore, this article has reference value for further research on artificial vision compensation imaging technology.

Figure 1

Pixelated imaging model framework based on locally saliency features
Pixelated imaging model framework based on locally saliency features

Figure 2

Example of directivity difference and Gabor filter detection result
Example of directivity difference and Gabor filter detection result

Figure 3

The percentage of full model experiment scores
The percentage of full model experiment scores

Figure 4

The average number of scores of imaging results weighted by significance
The average number of scores of imaging results weighted by significance

Single feature subjective scoring results statistics (mean ± standard deviation).

Feature Contrast Edge density Directional difference Symmetry
Human face 2.10±0.12 3.11±0.61 2.37±0.01 2.47±0.17
Typical objects 3.30±0.43 2.95±0.5 2.14±0.11 2.81±0.21
Indoor scene 2.88±0.17 2.69±0.17 2.24±0.08 0.41±0.17
Outdoor scene 3.07±0.17 2.24±0.11 2.19±0.08 2.88±0.19

Single-feature imaging results, single-factor analysis of variance results.

Scene type
human face Typical objects Indoor scene Outdoor scene
Null hypothesis probability 3.8×10−3 3.3×10−4 4.2×10−4 1.2×10−4

Fernández-Fontecha, A., O’Halloran, K. L., Tan, S., & Wignell, P. A multimodal approach to visual thinking: The scientific sketchnote. Visual Communication., 2019; 18(1): 5–29 Fernández-FontechaA. O’HalloranK. L. TanS. WignellP. A multimodal approach to visual thinking: The scientific sketchnote Visual Communication 2019 18 1 5 29 10.1177/1470357218759808 Search in Google Scholar

Kyurkchiev, N., Kyurkchiev, V., Iliev, A., & Rahnev, A. Some nonstandard differential models with applications to the population dynamics and computer viruses propagation. Dynamic Systems and Applications., 2019; 28(3): 757–788 KyurkchievN. KyurkchievV. IlievA. RahnevA. Some nonstandard differential models with applications to the population dynamics and computer viruses propagation Dynamic Systems and Applications 2019 28 3 757 788 Search in Google Scholar

Fehling-Kaschek, M., Peckys, D. B., Kaschek, D., Timmer, J., & de Jonge, N. Mathematical modeling of drug-induced receptor internalization in the HER2-positive SKBR3 breast cancer cell-line. Scientific reports., 2019; 9(1): 1–16 Fehling-KaschekM. PeckysD. B. KaschekD. TimmerJ. de JongeN. Mathematical modeling of drug-induced receptor internalization in the HER2-positive SKBR3 breast cancer cell-line Scientific reports 2019 9 1 1 16 10.1038/s41598-019-49019-x672214231481718 Search in Google Scholar

Triana, M., & Zubainur, C. M. Students’ Mathematical Communication Ability through the Brain-Based Learning Approach Using Autograph. Journal of Research and Advances in Mathematics Education., 2019; 4(1): 1–10 TrianaM. ZubainurC. M. Students’ Mathematical Communication Ability through the Brain-Based Learning Approach Using Autograph Journal of Research and Advances in Mathematics Education 2019 4 1 1 10 10.23917/jramathedu.v4i1.6972 Search in Google Scholar

Laptiev, O., Savchenko, V., Kotenko, A., Akhramovych, V., Samosyuk, V., Shuklin, G., & Biehun, A. Method of Determining Trust and Protection of Personal Data in Social Networks. International Journal of Communication Networks and Information Security., 2021; 13(1): 15–21 LaptievO. SavchenkoV. KotenkoA. AkhramovychV. SamosyukV. ShuklinG. BiehunA. Method of Determining Trust and Protection of Personal Data in Social Networks International Journal of Communication Networks and Information Security 2021 13 1 15 21 Search in Google Scholar

Nurdyansyah, N. Teaching Media Design Innovation Using Computer Application with Scientific Approach. International Journal of Academic Research in Business and Social Sciences., 2019; 9(3): 373–382 NurdyansyahN. Teaching Media Design Innovation Using Computer Application with Scientific Approach International Journal of Academic Research in Business and Social Sciences 2019 9 3 373 382 Search in Google Scholar

Lorenzo, G., Hughes, T. J., Dominguez-Frojan, P., Reali, A., & Gomez, H. Computer simulations suggest that prostate enlargement due to benign prostatic hyperplasia mechanically impedes prostate cancer growth. Proceedings of the National Academy of Sciences., 2019; 116(4): 1152–1161 LorenzoG. HughesT. J. Dominguez-FrojanP. RealiA. GomezH. Computer simulations suggest that prostate enlargement due to benign prostatic hyperplasia mechanically impedes prostate cancer growth Proceedings of the National Academy of Sciences 2019 116 4 1152 1161 10.1073/pnas.1815735116634769830617074 Search in Google Scholar

Pandey, G., & Ghanekar, U. Classification of priors and regularization techniques appurtenant to single image super-resolution. The Visual Computer., 2020; 36(6): 1291–1304 PandeyG. GhanekarU. Classification of priors and regularization techniques appurtenant to single image super-resolution The Visual Computer 2020 36 6 1291 1304 10.1007/s00371-019-01729-z Search in Google Scholar

Aghili, A. Complete Solution For The Time Fractional Diffusion Problem With Mixed Boundary Conditions by Operational Method. Applied Mathematics and Nonlinear Sciences., 2021; 6(1): 9–20 AghiliA. Complete Solution For The Time Fractional Diffusion Problem With Mixed Boundary Conditions by Operational Method Applied Mathematics and Nonlinear Sciences 2021 6 1 9 20 10.2478/amns.2020.2.00002 Search in Google Scholar

Rajesh Kanna, M., Pradeep Kumar, R., Nandappa, S. & Cangul, I. On Solutions of Fractional order Telegraph Partial Differential Equation by Crank-Nicholson Finite Difference Method. Applied Mathematics and Nonlinear Sciences., 2020; 5(2): 85–98 Rajesh KannaM. Pradeep KumarR. NandappaS. CangulI. On Solutions of Fractional order Telegraph Partial Differential Equation by Crank-Nicholson Finite Difference Method Applied Mathematics and Nonlinear Sciences 2020 5 2 85 98 10.2478/amns.2020.2.00017 Search in Google Scholar

Kusumah, Y. S., Kustiawati, D., & Herman, T. The Effect of GeoGebra in Three-Dimensional Geometry Learning on Students’ Mathematical Communication Ability. International Journal of Instruction., 2020; 13(2): 895–908 KusumahY. S. KustiawatiD. HermanT. The Effect of GeoGebra in Three-Dimensional Geometry Learning on Students’ Mathematical Communication Ability International Journal of Instruction 2020 13 2 895 908 10.29333/iji.2020.13260a Search in Google Scholar

Artículos recomendados de Trend MD

Planifique su conferencia remota con Sciendo