A Review of Virtual Surgical Object Modeling Technology Based on Force Feedback

Surface model based on surface mesh refers to many discrete particle by line connected together by the model, the model of topological relations between particle and particle, according to the topological relationship model can be further calculated the stress or deformation, therefore, choose the appropriate topology relationship is the premise of guarantee authenticity model deformation. Common topologies of face models include triangles, quadrangles, or other polygons, as shown in Figure 1. In the process of grid division, triangle structure is the smallest topological structure that can be divided. Compared with quadrilateral structure, there is no need to judge whether four points are on the same plane. Therefore, triangle structure is simple, flexible, easy to describe and efficient, and has been adopted by most scholars [7]. Early in virtual surgery system, most scholars use the surface of the triangular structure model to simulate the surgery simulation, for example, in 1994, Massie is simulated by the method, the operation objects in the virtual environment, and the operation object gives stiffness characteristics, force analysis through the spring model can calculate the size of the force feedback, and feedback to the user, The user can get a tactile feeling, but the model can not reflect the deformation characteristics. In 2017, Liao Denghong et al. used the grid deformation algorithm to realize the drilling operation of teeth, but the model could only reflect the changes of tooth surface, but could not reflect the structural changes inside the tooth model.

Figure 1.

Surface model based on surface mesh has certain advantages in displaying the surface structure of the model, and the modeling speed of the surface model is faster than that of the volume model, which can fully display the topological relationship between each particle on the surface of the surgical model. The model of single tooth surface based on triangular plane is shown in Figure 2.

Figure 2.

Volume model based on voxel refers to contain internal structure information of three-dimensional data model, the model consists of a number of individual element, voxel volume element, namely the voxel is through the body to a space according to certain size and manner of dividing, similar to the pixels, each individual element are closely linked and has its own location information, and fully displayed the model the internal organization of the information, Therefore, the selection of voxels is of great significance to the authenticity of model deformation. Common voxel structures include cubes, spheres or other shapes, as shown in Figure 3. Because cube is the easiest method for structure division, researchers mostly use cube structure to divide voxels and generate corresponding volume models [8, 9]. In 1997, Stijn Oomes applied Scale-space Theory to achieve voxelization of the model. In 1998, Roni Yagel et al. voxelized the model using a function of a normal vector. In 2000, Jones et al. used Distance Fields and Distance Transform to realize the voxelization of the model. Although these voxel methods can generate voxel models faster and better, they all need graphic workstations to participate in and have high requirements on hardware. Therefore, in 2004, Wu Xiaojun et al. proposed a method to voxelize the mesh model by using the structure of the octree. This method calculates the distance from the triangle to the center of the voxel. If the calculated distance is less than the preset threshold, it is considered that the voxel unit intersects with the triangle, and finally voxelization is realized. Although this method does not require the participation of a graphics workstation and only requires a computer, it consumes a lot of computing resources in the calculation process, and the threshold is difficult to determine, resulting in inefficient voxelization and incomplete voxel search. In 2010, Mu Bin et al. proposed a new voxelization method based on the octree structure. By calculating the projected volume, it is judged whether the voxel intersects the triangle, and the adjacency relationship is used to fill the interior of the model with voxels., and finally realize the voxelization process. This method projects the 3D to the 2D plane, and then returns the 2D data to the 3D space to realize the voxelization. Although the speed is improved, the processing process of this method is relatively Complicated, returning from 2D to 3D will cause errors to cause incomplete voxel search errors. In 2017, in order to improve this shortcoming, Duan Weiwei et al. proposed a new voxelization method based on the octree structure. First, the octree was used to subdivide the model, and then the model was scanned from multiple directions. This method Although it can quickly determine the surface voxels and internal voxels of the model, it is not suitable for the case where the model contains holes, and it will treat the internal holes as internal voxels. Therefore, the voxelization method still needs to be further improved.

Figure 3.

Compared with the surface model, the volume model based on voxel can show the structure of the internal information, for example in the oral cavity in virtual surgery, voxel model can according to different physical properties of the internal structure of simulated teeth of layered internal structure, such as enamel, dentin and dental pulp, etc., thus brings the user in the process of the whole operation and clinical surgery similar visual and tactile sensation. Therefore, the body model is used by most scholars to build the virtual surgical object model. The single tooth volume model based on cube is shown in Figure 4.

Figure 4.

At present, common modeling methods for soft tissue deformation include Mass Spring Method (MSM), Finite Element Method (FEM) and Boundary Element Method (BEM) [11]. The analysis and comparison of soft tissue deformation modeling methods is shown in Table I.

Mass spring method refers to the use of some discrete particles to describe the object, the particles and particles are connected through the spring, the force of the particle meets Newton's laws of motion, the spring meets Hooke's law. The mass spring model is shown in Figure 5.

Figure 5.

Refers to a continuum of finite element method is used to solve the domain of discrete into several units, each unit through the certain way of interconnecting approximation instead of the original system, and then within each unit, with the assumption of approximate function to piecewise said the whole solution domain and the unknown variables, finally through with the original problem, the mathematical model of the equivalent variational principle or weighted method, The equations of ordinary differential equations are established to solve the basic unknowns, and then numerical analysis is used to solve the equations, and finally the solution of the original problem is obtained.

Boundary element method refers to discretely solving the boundary of the solution domain by piecewise functions. This method is also a numerical analysis method, but it requires less computation than the finite element method. The boundary element model is shown in Figure 6.

Figure 6.

In order to enable users to get tactile feelings in virtual surgery, that is, users can obviously feel the contour, stiffness and other characteristics of the model surface through force feedback equipment [13], so as to make the experience more real, it is necessary to add mechanical properties to the geometric model and establish an appropriate feedback force calculation model for tissue and organ models. At present, the most commonly used force feedback model is the mass-spring-damper model. However, for rigid body models, such as teeth and human bones, their surfaces will not deform with the contact of the surgical tool model, so damping force is not needed to be considered.

The tactile feedback of the mass-spring-damper model is rendered by the relative position of the surgical tool model, In order to prevent the “piercing” phenomenon, that is, when the surgical object model is touched with the surgical tool, the surgical tool is prevented from being embedded inside the surgical object model. Therefore, for the situation where the visual and tactile positions of the surgical tool model are inconsistent, 1995 In 2008, Zilles and Salisbury proposed a point-based three-degree-of-freedom force feedback calculation method, also known as the God-Object method. This method saves two position information, one is the force contact point, which refers to the position where the surgical tool does not receive any resistance during the movement process, that is, the ideal position; the other is the God-Object point, when the surgical tool is in contact with the position of the collision point when the model object collides. When the surgical tool touches the surgical object in the process of moving, the force contact point will penetrate the surgical object, and the God-Object point will stay on the surface of the surgical object, and the feedback force can be obtained by calculating the distance between the two points. However, this method has a disadvantage. It is easy to cause the God-Object point to fall into the gap due to the error, resulting in discontinuity of force. Therefore, in 1997, Ruspini proposed a Virtual-proxy method based on this method. The Object point is replaced by a small ball without weight, which improves the disadvantage of falling into the gap and realizes the feedback effect of friction [14]. The idea of this method is the same as the God-Object method, require the force feedback Device to store two location information in the virtual surgical scene, one is the actual terminal location of the force feedback device, the other is the proxy location of the surgical tool model. In the initial state, the tool model has not collided with the object model. Therefore, the proxy and device overlap at this time. When the collision between the tool model and object model is detected, the proxy of the tool model will always be on the surface of the object model where the collision occurs. The device “pricks” into the object model as the force feedback device applies varying amounts of force. That is, visually, the tool model is on the surface of the object model, but tactile, the tool model is already deep inside the object model, as shown in Figure 7.

Figure 7.

When there is no collision in the moving process of the tool model, the feedback force is 0. When a collision occurs, the spring will stretch with the force applied by the force feedback device. The coordinate of the collision point should be found according to the line between the target position of the tool model and the position of the previous moment and the intersection of the object model surface. Then hooke's Law is used to calculate the size of the feedback force. Its feedback force can be described as formula (1). (1) F=(kΔx+dΔx)n→ {\rm F} = \left( {{\rm k}\Delta {\rm x} + {\rm d}\Delta {\rm x}} \right)\vec {\rm n}

Where k represents the elastic coefficient of the virtual spring, the distance between the device and the proxy of the surgical tool model, and d represents the damping coefficient, and n→ \vec {\rm n} represents the direction vector of the normal vector of the action point [15]. By adjusting k and d values, the mechanical properties of the surgical object in the actual operation can be accurately simulated, so that doctors can feel the feedback force in the real operation.

After a good model is established, if you want to get the tactile feeling in the virtual surgical system, you need to introduce force feedback devices. Now there are many force feedback devices on the market. Examples include Phantom Omni (Geomagic Touch) from Sensable, 3D Touch from Novint Falcon, Omega.6 and Omega.7 from Force Dimension, as shown in Figure 8. Phantom Omni force feedback device supports 3-Dof position and 3-Dof force feedback, 3D Touch force feedback device supports 6-Dof position and 3-Dof force feedback, Omega.6 force feedback device supports 6-Dof position and 3-Dof force feedback, Omega.7 force feedback equipment supports seven degrees of freedom position and three degrees of freedom force feedback. User through the manipulation of the force feedback device control handle control surgical tools and operation model to interact, the force feedback device access to the user by controlling the control handle to the operation model of the force f a state information, the state information including the tool position or size of the force, and then through the establishment of good physical model, Calculate the size and direction of the reaction force generated by this force, so as to generate force control signal, and then calculate the feedback force through the actuator in the force feedback device, and transmit it to the user through the control handle, so that the user can get the tactile feeling in the virtual environment.

Figure 8.