Research on the Estimation of Gaze Location for Head-eye Coordination Movement

The data collection experiment in this paper recruited 8 male graduate student volunteers as subjects, aged 23–30 years old, in good health and with good eyesight. Before the experiment, each subject was familiar with the specific content and precautions of the experiment, and they all participated in the experiment voluntarily. Each subject signed a written commitment and informed letter to ensure the legitimacy of the experiment in this paper. In order to exclude external interference, after the preparation for the experiment, each experiment was completed by only one subject alone.

The experimental equipment includes a DELL computer, an inertial sensor (MTI-G-700), three industrial cameras, three high-definition displays (resolution 2560× 1440), and eight infrared point light sources. Among them, the inertial sensor was worn to about the position of the occipital bone behind the subject's head to measure the Euler angles (Pitch, Roll, Yaw) of the subject's head posture when capturing the target. Since the MTIG-700 inertial sensor is only used to measure the change of the head posture in this paper, it will be referred to as the head posture instrument in the following. In this study, three Point Grey GS3-U3-41C6NIR-C industrial cameras were selected, and the resolution of the collected images was 2048×2048 and the chromaticity was near-infrared (NIR). Three cameras were installed above the three high-definition monitors, and were used to simultaneously capture the head motion images and eye motion images captured by the subjects during the experiment.

For the data collection experiment of head-eye coordination movement, this study innovatively built a non-wearable sight-drop data collection platform of “three eyes and eight light sources”. This platform not only expands the subject's head movement range, but also effectively reduces the impact of changes in lighting conditions. It is mainly composed of three industrial cameras mounted on three monitors and eight near-infrared light sources evenly distributed on the border of the monitors, which are used to record the head-eye images of the subjects when the target is captured. The schematic diagram of the platform deployment is shown in Figure 1.

Figure 1.

The head-eye movement data collection experiment designed in this study mainly refers to the process of the subjects performing visual interaction with the randomly appearing objects on the three screens through coordinated head-eye movement. Before starting the experiment, the infrared light source, camera, head attitude meter and other equipment should be calibrated to ensure that each equipment is in normal operation. The subjects were required to wear the head posture meter, and adjust the horizontal distance between the sitting position and the middle screen to save about 60cm to ensure that they were within the best focal length of the three cameras. At the beginning of the experiment, the subjects' eyes need to face the center of the middle screen, and press the record button to calibrate the initial Euler angle of the head posture. After the calibration is successful, the target to be captured appears randomly on the three screens in the form of a red circle with a radius of 30 pixels. The subject uses the head-eye movement to aim at the target, and press the record button to complete the target capture process.. During the experiment, there are no other requirements for the subjects, and the head can move freely in a large range. After the experiment, the program will record the Euler angle of the head pose, head image, eye image and the coordinates of the center point of the target each time the subject captures the target, and set it as a set of data. The experimental process is shown in Figure 2.

Figure 2.

Considering that the experimental operation is relatively simple, in order to ensure the experimental status of the subjects and the quality of the experimental data, the duration of a single experiment is set to 20 minutes in this study. During the experiment, the equipment was deployed on a six-axis full-motion simulated flight platform, as shown in Figure 3. Due to the long-term use of the camera, the performance will be effectively degraded, and there may be cases of missed shots, so simple manual screening is required after each experiment. Finally, after screening unqualified samples, a total of 31507 groups of head-eye movement data were collected in this paper.

Figure 3.

In this paper, the posture measuring instrument was worn on the back of the subject's head and used as the three-axis reference point for head movement. The head attitude data mainly includes: pitch angle (Pitch), yaw angle (Yaw), roll angle (Roll), namely looking up, shaking head and turning head, through these three Euler angles, the head position can be estimated more accurately. space pose. In this study, a right-handed Cartesian coordinate system is used, and the three-axis positions of X, Y, and Z in space and the corresponding Euler angles of the head posture are shown in Figure 4.

Figure 4.

In order to facilitate the statistics and analysis of the head pose data, this study visualized the recorded data. The three-axis pose data of the head is shown in Figure 5. Through the three-axis Euler angle deflection angle, it can be intuitively observed that the head yaw angle (Yaw) and the pitch angle (Pitch) change greatly during the target acquisition process, while the roll angle (Roll) changes less.

Figure 5.

The human eye detection method used in this study mainly uses the AdaBoost cascade classifier combined with Haar-like features to first detect the face area, and then detects and intercepts the human eye area in the face area. Based on the pre-trained classifiers for faces, eyes, etc. included in OpenCV, this study carried out face detection and eye detection on the front view image collected by the camera corresponding to the target appearing screen. The recognition results are shown in Figure 6. According to the results of human eye detection, the monocular area of the subject is intercepted at a resolution of 64×64, and the left and right eye images are obtained as the input of the next convolutional neural network model.

Figure 6.

Aiming at the requirement of lighting conditions in appearance-based visual estimation method, this study combines the idea of feature-based visual estimation method, and places eight near-infrared point light sources equidistantly on the boundary of three screens. The Purkinje formed by the reflection are used as the feature points of the eye image. Since the camera is a near-infrared camera, the brightness of the Purkinje formed by the reflection of the infrared point light source through the corner of the eye is not affected by external light. To sum up the above assumptions, this study performed threshold processing on the intercepted left and right eye images before training the line of sight placement model, and obtained left and right eye images with more obvious Purkinje spots, which were used as the control group input by the convolutional neural network model. Its influence on the estimation result of the line-of-sight landing point.

In this study, the three monitors (resolution: 2560×1440) are numbered in the order of left (0), middle (1), and right (2). degree. This study takes the five target points on the left (0) screen as an example for analysis, and the specific positions are shown in Figure 7. Among them, the first two coordinates of each point are the position of the target center pixel on the screen, and the third coordinate is the screen number.

Figure 7.

The eye images collected at the five points A, B, C, D, and E in the above figure are processed in the order of binarization thresholding, truncation thresholding, and super-thresholding zero. The detection effect is shown in Figure 8. It can be observed from the figure that for different fixation points, the number and positional relationship of Purkinje spots formed by the subjects' eyes are different. Treatment can detect 6–7 Purkinje. Among them, the Purkinje after the truncated thresholding process is more obvious, which can effectively eliminate the reflected light spots on the cornea of other external light sources, and retain the original eye image. Therefore, this paper will use the truncation thresholding method to process the cropped left and right eye images, and detect the left and right eye images with obvious Purkinje as the input of the next convolutional neural network model.

Figure 8.

For the deep convolutional neural network model, this paper refers to the deep residual network structure proposed by researchers such as He in 2015. At present, several commonly used ResNet networks mainly include: ResNet-18, ResNet-34, ResNet-50 and other variants. Although increasing the depth of the network can improve the accuracy of the model, the shallower residual network (ResNet-18) also has good accuracy in practical applications, and its model is small, which provides faster convergence speed and facilitates parameter optimization. Moreover, based on the short-circuit operation of the ResNet model, the combination of features of different resolutions can be realized, and it has a better feature extraction effect for the eye image input in this paper. Therefore, in this study, ResNet-18 is used as the estimation model of the human eye line of sight, and the network structure of ResNet-18 is shown in Table 1. By comparing different types of binocular image inputs (with and without thresholding to detect Purkinje), analyze the accuracy of the output on the screen where the sight falls. Since traditional residual neural networks are mostly used for classification tasks, this paper is inspired by the Google team's research on line-of-sight drop estimation in 2019. Multiple fully-connected layers are connected after the hidden layer of the neural network to return the line-of-sight drop coordinates.

For the estimation model of human eye gaze point in this paper, the input is the grayscale images of the left and right eyes when the subject is facing the capture target in the middle of the screen. Due to the low resolution requirement of the eye image, this paper adopts the resolution size of 64×64 to capture the monocular image. Compared with the input size of the traditional ResNet-18 network RGB image (224×224×3), the image input in this paper is smaller (64×64×1), which improves the computational speed of the model. This model is divided into two branches with the same structure. The input layer is the cropped left and right eye images, and the main structure of the hidden layer of each branch is built according to the ResNet-18 network structure. It consists of 17 convolutional layers, 8 residual blocks and 2 pooling layers. Relu is used as the activation function of all convolutional layers to achieve feature extraction for left and right eye images. Finally, the left and right eye images are extracted through 4 fully connected modules. The feature maps of the eyes are fused and the estimated line-of-sight coordinates G(x, y, n) are output. Figure 9 shows the structure of the network model for the estimation of the human eye gaze point in this paper.

Figure 9.

In order to fuse the head pose data in the line-of-sight drop estimation, this study draws on the idea of using the traditional BP neural network, and uses multiple fully connected layers as the network branch of the head pose data feature extraction, mainly including: an input layer and a Hidden layer composition. Among them, the input layer is the subject's head posture Euler angles (Roll, Pitch, Yaw); the hidden layer is composed of three fully connected layers, the number of neuron nodes is 100, 16, 16 respectively, and Relu is used as the activation function uses the feature vector extracted by the last fully connected layer as the output.

Based on the principle of feature layer fusion, this research first preprocesses the head-eye data to complete feature extraction. Image feature extraction; for the head image, the Euler angle of the head pose is used to output the feature vector with the same dimension as the eye feature through the head feature extraction network to complete the dimension registration. The features of the two parts are fused through multiple fully connected layers to form a line-of-sight estimation model structure fused with head-eye movements. The network structure of the line of sight estimation model in this study includes three branches. The input layer inputs the left and right eye images and the Euler angle of the head pose when the subject is capturing the target. For the left and right eye branches, three fully connected layers are used for feature extraction, and the number of neuron nodes is 128, 32, and 16 respectively; for the head pose branch, a head feature extraction network is used, and finally two neurons are used for feature extraction. The fully connected modules with the number of nodes are 16 and 3 to complete the feature fusion of the data of the three branches, and realize the regression of the landing point of the three screens, as shown in Figure 10.

Figure 10.

头眼协调运动是协调组合头部、眼部运动并 合成统一动作从而完成视线转移向目标的过程 [3]。在视觉落点估计的研究中, 需要建立头部 运动、眼部运动和视线移动的关系, 才能得到 较为准确的视线落点位置。因此, 如何建立头 眼-视线关系是研究的关键问题。

对于头眼-视线关系的研究, 最初的方式是 通过限制人的头部自由运动, 仅对眼部运动进 行跟踪[4], 使得头部稍有移动便造成很大的系 统误差, 且不适应于实际应用场景。为此, 头 眼运动机制成为了当时研究的热点。2008 年, Freedman E G[5]使用生理学方法对与人类头眼 运动机制相似的猕猴作为研究对象, 测得视线 移动与头眼运动的关系。其实验结果表明了头 眼运动与视线的关系：当目标出现于较大视野 范围中时, 双眼会率先于头部朝目标进行移 动, 随后头部开始同向移动。由于双眼运动速 度较快, 视线可快速地完成目标捕获并停止运 动。但头部运动较为缓慢, 任未停止向目标方 向移动。此时, 在前庭功能的作用下, 趋势双 眼以一定的速度反向移动, 作为补偿头部运动 的 前 庭 动 眼 反 射 (vestibulo-ocular reflex, VOR)[6], 保证目标稳定地 存在于视线范围 内。由此可得, 虽然头部运动对目标捕获过程 的贡献较小, 但其也直接影响了视线移动的方 向。

而视线估计是利用机械、电子、光学等现有 检测技术对受试者当前视线方向或视线落点的 研究[7]。早期的视线估计研究得益于医学与心 理学的发展, 研究者多以直接观察的方式记录 眼睛运动的相关信息。基于视线估计所使用设 备的不同, 可将视线估计研究划分为穿戴式与 非穿戴式[8]。随着视线估计技术的发展, 出现 了如接触镜(Contact Lens)和眼电图(EOG)等穿 戴式的视线估计方法, 虽然减小了头部运动的 影响, 但对受试者干扰较大, 不适于较长时间 的佩戴。伴随图像处理与计算机视觉技术的发 展, 使得基于视频的视线估计方法便捷、非穿 戴式的优势逐渐显现出来, 并在医疗诊断、辅 助驾驶和人机交互等多个领域得到普及与应 用。但对于视线估计研究而言, 如何高效地融 入头部运动数据仍然有待进一步的研究。

近些年, 在深度神经网络的研究热潮下, 基 于头眼数据融合的视线估计算法研究有了新的 进展, 主要可分为对注视目标的估计、视线落 点估计与视线方向估计。2016 年, Recasens[9] 等人设计了一个由两支路组成的深度神经网络 模型, 分别用于提取人物图像的头部姿态与注 视方向, 通过目标显著性判断实现对注视目标 的估计；Kyle Krafka[10]等人以手机和平板电 脑等设备为研究对象, 设计了由四个支路构成 的深度神经网络, 分别输入左右眼图、人脸图 像和人脸位置, 实现了于二维平面的视线落点 估计。此研究对左右眼图像支路的参数权值进 行共享处理的思想, 受到此后很多研究的借 鉴；2019 年, Google[11]团队对上述模型做了 进一步改进, 将视线落点估计模型改为三支路 并由四个眼角的坐标位置代替人脸图像与人脸 位置, 最终取得了不错的视线落点估计效果； 对于视线方向的估计, 通常由水平和竖直两个 角度所构成方向向量表示, Zhang[12]于 2015 年的研究中, 将输入图像的头部姿态数据与眼 部特征进行拼接, 使用类似于 LeNet[13]的浅 层网络结构实现对视线方向的估计, 此研究对 头眼数据融合的方式对本文后续研究起到了很 大启发。

本研究采用基于外观的视线估计方法, 通过 非穿戴式多目相机所拍摄的头眼图像, 实现对 被试的视线落点估计。虽然基于外观的视线估 计方法有较强的鲁棒性, 但也存在着对被试头 部自由运动限制较大、受光照条件变化影响较 大等问题。而本文搭建的“三目八光源”平台 通过三部显示器的捷联, 不仅可以扩大被试的 头部运动范围；并通过红外光源在人眼角膜上 的反射光斑（普尔钦斑）, 为眼部图像增加了 特征点, 减小光照的影响。为此, 本研究通过 图像处理与特征提取的方法, 融合头部特征与 眼部特征, 建立视线估计神经网络模型, 进而 实现视线落点估计的研究。

1)

头部特征提取

本文将姿态测量仪佩戴至被试的后脑勺位 置, 并将其作为头部运动的三轴基准点。头部 姿态数据主要包括：俯仰角 (Pitch)、偏航角 (Yaw)、滚转角(Roll), 即抬头、摇头和转头, 通过这三个欧拉角可以较为准确地估计出头部 的空间姿态。本研究采用右手笛卡尔坐标系, 其空间 X、Y、Z 三轴位置与对应的头部姿态 欧拉角如图 4 所示。

图 4

为了便于对头部姿态数据的统计和分析，本研究对所记录数据进行了可视化处理，头部的三轴姿态数据如图5所示。通过三轴欧拉角偏转角度，可直观地观察出头部偏航角(Yaw)和俯仰角(Pitch)在目标捕获过程中的变化幅度较大，而滚动角(Roll)的变化幅度较小。

图 5

2)

眼部特征提取

本研究使用的人眼 检 测方法主要是利 用 AdaBoost 级联分类器结合 Haar-like 特征先检 测出人脸区域, 再在人脸区域中检测出人眼区 域并进行截取。基于 OpenCV 所包含针对面 部、眼部等进行过预训练的分类器, 本研究对 目标出现屏幕对应相机所采集的正视图像进行 了人脸检测、人眼检测, 识别结果如图 6 所 示。通过人眼检测结果, 对被试的单眼区域按 64×64 的分辨率进行截取, 得到左右眼部图像 作为下一步卷积神经网络模型的输入。

图 6

针对基于外观的视觉估计方法所存在对光照 条件的要求, 本研究结合基于特征的视觉估计 方法的思想, 在三个屏幕的边界等距地安放了 八个近红外点光源, 通过其在角膜的反射所形 成的普尔钦斑作为眼部图片的特征点。由于相 机属于近红外相机, 因此红外点光源经眼角反 射所形成普尔钦斑的亮度不受外界光照影响。 综上述设想, 本研究在对视线落点模型训练 前, 对所截取的左右眼图像进行阈值处理, 得 到含有较为明显普尔钦斑的左右眼图像并作为 卷积神经网络模型输入的对照组, 对比其对视 线落点估计结果的影响。

本研究对三台显示器(分辨率：2560×1440) 按左(0)、中(1)、右(2)的顺序编号, 为了更加 直观地对比几种阈值处理的效果与普尔钦斑的 明显程度。本研究以左(0)屏的五个目标点位为 例进行分析, 具体位置如图 7 所示。其中, 每 个点前两个坐标为目标中心像素点于屏幕上的 位置, 第三个坐标为屏幕号。

图 7

对上图 A、B、C、D、E 五点的所采集的眼 部图像按照二值化阈值、截断阈值化和超阈值 化零的顺序进行处理, 对比几种阈值处理方法 对普尔钦斑的检测效果, 其结果如图 8 所示。 由图中可观察出, 对于不同的注视点, 被试眼 部所形成的普尔钦斑的数量与位置关系都有所 不同, 当注视左屏(0)屏幕中心位置时(C), 通 过阈值处理可检测出 6-7 个普尔钦斑。其中, 截断阈值化处理后的普尔钦斑较为明显, 能有 效地消除了其他外界光源的在角膜的反射光 斑, 保留了原始眼部图像。因此, 本文将采用 截断阈值化处理法对裁剪后的左右眼图像进行 处理, 检测含有明显普尔钦斑的左右眼图像作 为下一步卷积神经网络模型的输入。

图 8

对于深度卷积神经网络模型, 本文参考了 2015 年由何凯明等研究者提出的深度残差网络 结构。目前, 常用的几种 ResNet 网络主要包 括：ResNet-18、ResNet-34、ResNet-50 以及其 他变种, 虽然增加网络的深度可以提升模型的 准确率, 但较为浅层的残差网络(ResNet-18)在 实际应用中同样有良好的准确性, 同时其模型 较小, 提供了更快的收敛速度, 便于参数的优 化。而且基于 ResNet 模型短接的操作, 可实 现对不同分辨率特征的组合, 对于本文输入的 眼部图像有较好的特征提取效果。因此, 本研 究以 ResNet-18 作为人眼视线落点估计模型, ResNet-18 网络结构如表 1 所示。通过对比不 同类型的双眼图像输入（有无通过阈值处理检 测普尔钦斑）, 分析其输出位于屏幕上视线落 点的精度。由于传统的残差神经网络多用于分 类任务, 本文受 2019 年 Google 团队对视线落 点估计的研究启发, 在神经网络的隐含层后连 接多个全连接层, 用于回归出视线落点坐标。

本文的人眼视线落点估计模型, 输入为被试 正视屏幕中带捕获目标时的左、右眼部灰度图 像。由于眼部图像的分辨率要求较低, 本文采 用 64×64 的分辨率大小截取单眼图像。相对 于传统 ResNet-18 网络 RGB 图片的输入大小 (224×224×3), 本文的图像输入较小(64×64 ×1), 使得模型的计算速度提高。而本模型分 为两条结构相同的支路, 其输入层为裁剪后的 左、右眼图图像, 而每条支路隐含层的主要结 构是按着 ResNet-18 网络结构搭建的, 主要由 17 层卷积层、8 个残差块构成和 2 个池化层构 成, 使用 Relu 作为所有卷积层的激活函数, 实 现对左右眼图像的特征提取, 最后通过 4 个全 连接模块将左右眼的特征图进行融合并输出所 估计的视线落点坐标 G(P_x, P_y, S_n), 本文的 人眼视线落点估计网络模型结构如图 9 所示。

图 9

3)

融合头眼的视线落点估计

为了在视线落点估计中融合头部姿态数据, 本研究借鉴采用传统的 BP 神经网络的思想, 通过多个全连接层作为头部姿态数据特征提取 的网络分支, 主要包括：一个输入层和一个隐 含层构成。其中, 输入层为被试的头部姿态欧 拉角(Roll, Pitch, Yaw)；隐含层由三个全连接层 组成, 神经元节点个数分别为 100, 16, 16, 并使用 Relu 作为激活函数, 将最后一层全连接 层所提取的特征向量作为输出。

基于特征层融合的原理, 本研究先对头眼数 据进行预处理以完成特征提取, 对于眼部图像 由上文中所提及的眼部视线落点估计模型的输 入层和隐含层实现左右眼部图像特征的提取； 对于头部图像则是由头部姿态欧拉角, 通过头 部特征提取网络输出与眼部特征维度相同的特 征向量, 完成维度配准。两部分特征通过多个 全连接层实现特征融合, 构成融合头眼运动的 视线落点估计模型结构。本研究的视线落点估 计模型的网络结构共包括三个支路, 输入层分 别输入被试在捕获目标时的左右眼图像与头部 姿态欧拉角。对于左右眼支路, 使用三个全连 接层进行特征提取, 神经元节点个数分别为 128、32、16；对于头部姿态支路, 采用头部 特征提取网络, 最后通过由两个神经元节点个 数为 16 和 3 的全连接模块完成对三条支路数 据的特征融合, 实现对三屏实现落点的回归, 如图 10 所示。

图 10

为了使人眼视线落点估计模型在学习中朝向 更精确的估计性能改进, 本文使用均方方差 (MSE)作为损失函数, MSE 表示预测数据与原 始数据所对应点误差的平方和均值： (1) MSE=1n∑i=1mωi(yi−y^ i)2 MSE = {1 \over n}\sum\limits_{i = 1}^m {{\omega _i}{{\left( {{y_i} - {{\hat y}_i}} \right)}^2}}

其中, n 是样本的个数, y_i 是原始数据, ŷ_i 是预 测数据。当 MSE 值越接近 0 时, 说明模型的 拟合能力越强, 其视线落点估计也越准确。基 于本文的三屏实验平台, 每个屏幕为一个二维 平面, 采用欧式距离计算标定点与估计点间的 差值, 预测接受域为以标定点为圆心, 半径 30 像素的圆形区域。在模型的训练过程中, 本文 使用自适应矩估计(Adam)作为优化器, 其能对 不同的参数调整不同的学习率；网络的学习率 设定为10⁻³, 批处理量设置为 32, 训练周期为 100。

本研究通过对原始数据的筛选, 去除被试闭 眼的图像后, 从中共选取了 30000 张图像并将 它们裁剪为适合模型输入的尺寸, 其中 70%用 于模型训练, 30%用于模型的性能测试。同 时, 以输入原始数据前是否进行普尔钦斑检测 的两种情况, 构成两种不同的视线落点估计模 型并作为对照实验, 对比分析最终的预测准确 率和其他性能。

实验结果表明, 当仅使用原始眼部图像作为 模型输入时, 模型(Eye)所需提取的特征较少, 收敛速度较快。在 200 个 epoch 左右模型基本 收敛, 其平均准确度可达到 85.6%；而当输入 为普尔钦斑检测后的眼部图像时, 虽然模型 (Eye&Purkinje) 的 收 敛 速 度 较 慢, 在 400 个 epoch 左右基本收敛, 但是其平均准确率可达 到 87.7% 。 通 过 对 比 两 种 模 型 的 性 能, Eye&Purkinje 模型在输入前通过阈值处理使普 尔钦斑特征更加明显, 增加了隐含层特征提取 的复杂度, 从而增加了模型的收敛时间, 但其 相对于 Eye 模型的视线落点估计平均准确率提 升了 2.1%, 且损失曲线较为平稳, 模型的稳定 性更好。因此, 本研究的选用 Eye&Purkinje 模 型作为人眼视线落点估计的模型, 并验证了输 入含有显著普尔钦斑的眼部图像, 可为图像增 加特征点, 减小光照条件的影响, 提高模型估 计的准确率。两种模型训练的准确率和损失变 化曲线, 如图 11 所示。

图 11

为了评价融合头眼运动的视线落点估计模型 的性能, 本文将其与仅使用眼部图像的两种模 型进行了对比分析。如图 12 所示, 在前 200 个 epoch 中, 加入头部姿态后的模型性能已优 于 Eye&Purkinje 模型, 但由于需要融合头眼特 征, 参数优化所需时间较长, 准确率不如 Eye 模 型 。 从 模 型 的 收 敛 速 度 分 析, 虽 然 Eye&Purkinje&Head 模型收敛较慢, 但在 600 个 epoch 后模型趋于平稳, 模型的精度较高, 其平均准确率可达 89.9%, 相对于仅使用单维 度的眼部图像输入模型, 该模型通过融合头眼 多维度的数据, 实现对头眼协调运动数据的压 缩与关联, 其准确率最多提升了 4.3%。

图 12

对于视线落点估计的研究, 本文分别采用了 三种模型进行了对比分析, 即仅使用眼部图像 的视线落点估计模型、使用眼部特征的视线落 点估计模型与融合头眼运动特征的视线落点估 计模型。通过对比, 融合头部运动与眼部特征 点图像的视线落点估计模型(EPH)对测试集的 预测结果有较高的测试准确度, 所估计的视线 落地基本在待捕获目标的预测接受域内且未出 现过拟合的现象, 其对视线落点估计的平均准 确率可达 89.9%。但本文也发现这三个模型普 遍估计的精度不高, 该问题是基于外观的视线 落点估计方法存在的普遍问题, 有待后续进一 步的研究与展望。

综上, 本文基于一种外观与特征相结合的视 线估计方法, 有效地融合了视线移动时的头部 运动与眼部运动, 将头部姿态数据与眼部特征 通过深度卷积神经网络结构最终实现了对二维 屏幕中视线落点的精确估计。为视线落点估计 研究提出了一种切实有效的研究方法。