Calibration of Structured Light Scanning System

Selection requirements: This project is expected to control the average error of the dental model measurement to 0.05mm, the camera measurement range is 110mm×90mm, the camera to the object distance is 200~350mm, and the measurement depth of field is required to be 50mm.

CCD camera pixel resolution: The resolution of the captured image is determined by the camera resolution and the size of the target scene. In this measurement system, the resolution of the CCD camera in the horizontal and vertical directions should be 110mm/0.05 respectively. Only when mm=2200 and 90mm/0.05mm=1800 can meet the measurement standards.

The three-dimensional point cloud measurement system can ignore the color and texture characteristics of the tooth surface, and the dental model measurement system has certain requirements for noise and transmission speed. Therefore, in order to introduce as little noise as possible to increase the transmission speed, two models are selected in this article It is a CCD series USB3.0 camera of Hikvision MV-CE100-30M. The resolution of the camera is 3840×2748, the maximum frame rate is 7.1 fps, and the target surface size is 1/2.3 inches (6.41mm×4.59mm).

System magnification:

Lens focal length:

Normally, it is required that the size of the target surface of the lens should not be smaller than the size of the camera’s target surface. This topic chooses ZX-SF1620C, the target surface size is 1/3 inches, the focal length is 16mm, the TV distortion of the lens is less than 0.1%, and the aperture is F1.4. ~F16C.

The clarity of the projection of the projector directly affects the accuracy of the measurement, and the image of a general projector is not clear within 110mm×90mm. This measurement system selects the micro projector optical machine with LCoS projection technology. The optical machine projection control chip model is Texas Instruments DLP3010EVM-LC. The projector luminous flux is 50 lumens, the projection size is 5-200 inches, and the projection distance is 100mm~2500mm. The pixel resolution is 1280×720, and the micro-mirror pitch is 5.4um. Experiments show that it can clearly project in the range of 110mm×90mm at a distance of 200~400mm. DLP3010EVM-LC supports the burning of programming graphics. Burn the raster pictures of the specific frequency needed during the measurement, the image capture card will save the images taken by the camera and transmit them to the computer. The turntable control module is connected to the computer and cooperates with the rotating platform to perform high-precision object postures for the rotation angle.

The angle between the two CCD cameras is about 30°, the angle between the single camera and the projector is 15°, and the actual working distance is about 317mm. The measurement software part mainly completes the system calibration and the resolution of the grating structured light. Combining the results of the camera calibration and the rotation center calibration module, the software system can solve the three-dimensional information of the object surface under a certain angle of view, and finally according to the rotation direction and rotation of the turntable The angle unifies the measurement results under the same coordinate system, and obtains the surface profile information of the object.

Figures 1 how the physical objects of the in vitro dental model measurement system. The operation process of the instrument is as follows: adjust the angles of the two cameras until the calibration plate is fully exposed in the field of view of the left and right cameras, project the grating structured light onto the dental model to be tested, and the rotating platform drives the dental model to start rotating at a certain angle Complete the scan in one week. The left and right cameras collect data information, and the obtained information is processed by a computer to obtain a three-dimensional model of the tooth.

Ideally, the camera’s imaging process is small hole imaging (as shown in Figure 2), but subject to the processing conditions of the lens, in fact, the light will shift to a certain extent during the process of passing through the lens, resulting in the mapping of three-dimensional objects in two-dimensional Distortion appears on the image. In order to correctly restore the position information of the measured object, the camera distortion must be corrected. The process of correcting the camera distortion is the camera calibration. Camera calibration technology has been studied for decades at home and abroad. Generally, it is necessary to design corresponding calibration methods and calibration devices for different measured objects and design an ingenious calibration scheme to improve the overall measurement accuracy of the system. The stability of the system is of great significance [4].

Figure 2.

• Coordinate system

The optical imaging system of the camera uses a convex lens to map the image of a three-dimensional object in reality onto a two-dimensional plane, as shown in Figure 3 This process requires the conversion of three coordinate systems. The three coordinate systems are: world coordinate system, camera coordinate system, and image pixel coordinate system. The world coordinate system is considered to be the coordinates of the real three-dimensional space where the measured object is located. The camera coordinate system is a coordinate system established with the camera’s optical center as the circle point and the camera’s optical axis as the Z axis. The image coordinate refers to a single pixel.

Figure 3.

• Coordinate system

The optical imaging system of the camera uses a convex lens to map the image of a three-dimensional object in reality onto a two-dimensional plane, as shown in Figure 3 This process requires the conversion of three coordinate systems. The three coordinate systems are: world coordinate system, camera coordinate system, and image pixel coordinate system. The world coordinate system is considered to be the coordinates of the real three-dimensional space where the measured object is located. The camera coordinate system is a coordinate system established with the camera’s optical center as the circle point and the camera’s optical axis as the Z axis. The image coordinate refers to a single pixel.

As shown in Figure 6, assuming that the point W(Xw,Yw,Zw) is a point in the three-dimensional space, the point of its projection on the imaging plane is p(u,v), (x,y), (xw,yw,zw) and (xc,yc,zc) respectively represent the three-dimensional coordinates of the point p in the world coordinate system and the camera coordinate system coordinate. Suppose the projection of the origin of the camera coordinate system in the imaging plane coordinate system is (u0,v0), sx, sy denote the number of pixels per unit distance, and f denote the effective focal length of the system. There is a perspective relationship:

(1) [ u v 1 ] = [ s x 0 u 0 0 s y v 0 0 0 1 ] [ x c y c 1 ]

The relationship between image coordinates and camera coordinates is:

(2) Z c [ x y 1 ] = [ f 0 0 0 f 0 0 0 f ] [ X c Y c Z c ]

The relationship between camera coordinates and world coordinates is shown in formula (3), R in the formula represents the rotation matrix, and represents the translation matrix:

(3) [ X c Y c Z c ] = [ R T ] [ X w Y w Z w 1 ]

The pixel coordinates to world coordinates are sorted out as shown in formula (4):

(4) Z c [ u v 1 ] = [ f × s x 0 u 0 0 f × s y v 0 0 0 1 ] [ r 1 r 2 r 3 t x r 4 r 5 r 6 t y r 7 r 8 r 9 t z ] [ X w Y w Z w 1 ]

The above formula can be converted as:

(5) Z c [ u v 1 ] = [ f × s x 0 u 0 0 f × s y v 0 0 0 1 ] [ R T 0 0 ] [ X w Y w Z w 1 ] = M 1 M 2 X

Among them, the parameters of the x matrix and the actual focal length matrix and the actual focal length, the distortion coefficient is related to the coordinates of the image center (principal point) and the angle between the camera and the lens. These are the internal parameters of the camera and have nothing to do with the movement of the camera and the pose of the camera. Therefore, the M ₁ matrix is also called the internal parameter matrix. The important parameters of M ₂ are the rotation matrix and translation matrix of the camera. The solution of these two matrices is related to the movement of the camera and the position and angle of the camera. These are the non-internal parameters of the camera, the matrix is called the external parameter matrix.

• Coordinate system

Ideally, the camera is a pinhole model, but due to component manufacturing and processing technology, the optical imaging components of the camera often have nonlinear geometric distortions. In the 3D reconstruction process, if you do not consider correcting the camera distortion, then you cannot Restore the true coordinate value of the measured object, resulting in measurement failure. Camera distortion models can generally be divided into two types: radial distortion and tangential distortion. Radial distortion shows that all light rays are closer to the center of the image, so it is also called pincushion distortion, while tangential distortion is the opposite. The light will deviate to the surroundings. Also called barrel distortion, the two distortions are shown in Figure 4.

Figure 4.

The radial distortion model of the camera can be expressed as:

(6) { x u = x d ( 1 + k 1 r d 2 + k 2 r d 4 + … + k i r d 2 i ) x u = y d ( 1 + k 1 r d 2 + k 2 r d 4 + … + k i r d 2 i ) ( i = 1 , 2 , 3 … )

The tangential distortion model of the camera can be expressed as:

(7) { x v = x r + [ 2 p 1 y r + p 2 ( r 2 + 2 x 2 ) ] y v = y r + [ 2 p 2 x r + p 1 ( r 2 + 2 y 2 ) ]

Unlike tangential distortion, radial distortion may need to be considered for higher orders but considering higher-order radial distortion coefficients may cause the calculation results to not converge and obtain high-precision calibration results. The calculation also increases the difficulty of solving. In actual measurement, only second-order or third-order distortion can be considered in camera calibration.

Zhang proposed a non-linear optimization method for camera calibration. The camera calibration can be completed by shooting the checkerboard pattern at multiple angles. The algorithm only needs to print a specific checkerboard pattern to achieve high-precision calibration of the camera., Low cost and simple to implement. However, in the process of using this method, the characteristic points on the calibration board, that is, the corner points between the checkerboards, must be accurately extracted, and the extracted points must be arranged in a certain order.

In order to ensure the accuracy of corner extraction, high-quality images are usually required. However, in practical applications, considering the cost factor, the quality of the camera and the calibration board are often uneven, making it possible to rasterize during the calibration process, leading to deviations in corner recognition, as shown in Figure 5.

Figure 5.

In order to solve the rasterization problem of the checkerboard calibration board, Xia Renbo et al. proposed a fully automatic camera calibration method based on circular marking points. The essence of this algorithm is to improve the corner detection to the center detection of the circular mark. The circular mark points have lower requirements for image quality and stronger anti-interference ability, but the mark point sorting process of the calibration algorithm proposed by Xia Renbo is more cumbersome. Liang Li et al. proposed a design plan for a plane calibration board with a progressive circle as the primitive. The checkerboard calibration template was retained, and the detection and sorting algorithm for the characteristic points of the circle was given. However, when the plane of the calibration target is relative to the optical axis of the camera, there is a risk of failure of the automatic sorting algorithm when it exceeds 90°. Hou Junjie and others proposed a calibration scheme based on concentric circles. The advantage of concentric circles is that the concentric circles in the world coordinate system are still concentric circles when they are mapped to the image coordinates, which can better ensure the accuracy of the center of the circle, and finally use the double Obtain the camera parameters based on the constraint relationship of the eye vision [8].

In view of this, this paper proposes an automatic detection and matching algorithm for the feature points of the calibration plate based on 4 concentric circular mark points. With Zhang’s camera parameter calculation method, it can realize the automatic calibration of the camera. The calibration board used in this paper is shown in Figure 6. The diameter of the big circle in the calibration board is 9mm, the diameter of the small circle is 3mm, and the interval between the small circles is 12mm. The parameter is the number of circular marker points along the X direction is 17, the number of circular marker rows along the Y direction is 14, and the four concentric circles in the sign orientation are as follows:

Figure 6.

The calibration process based on circular marking points is shown in the Figure 7:

Figure 7.

In order to facilitate the extraction of the dots directly through threshold segmentation after the image is collected, the calibration plate uses a black background and white pattern, and the surface is treated with aluminum oxide to prevent reflection. The physical picture of the calibration plate is shown in Figure 8.

Figure 8.

Since the dot will become an ellipse after the perspective transformation, the ellipse detection method can be used to obtain the set P of ellipses in the calibration plate.

The four large circles on the calibration board are arranged as shown in Figure 5. According to the semi-major axis of the ellipse, the positions of these large circles can be measured as, and the next step is to identify these mark circles and number them:

a) Find the two closest circles in E and mark them as E1 and E2 (they correspond to III, IV or IV, III);

b) Find the great circle closest to the line where E1 and E2 are located among the remaining two circles, and record it as E3( II ), then the remaining E4 is circle I;

c) Then the great circle closest to II is III.

As shown in Figure 8, the dots at the four corners of the calibration plate are A, B, C, D, and the matching relationship of all dots can be determined according to the following method:

According to the positioned ellipses I and II, the point on the left side of the line and the farthest distance from the line in the calculation P is point A. Similarly, the position of the BCD point can be determined. The direction and position relationship between the calculated point and the straight line is as follows: If c(x,y), s(x_s,y_s ), and e(x_e,y_e ), is 3 points on the plane, the judgment formula of the azimuth relationship between c and the straight line is:

(8) F = y ( x e − x s ) − x ( y e − y s ) + x s y e − x e y s

In formula (8), the observation is from the point to, if the point is on the left, the calculation result F>0, otherwise, it is considered to be on the right of the observer. This method can cleverly judge the point on the calibration board the azimuth relationship with the great circle.

The sorting method of circular marker points in this article is as follows:

Step1: Find the point on the straight line AB. This step can be considered as searching for the 14 mark points that are closest to the straight line where AB is located in the set of all points p, denoted as P _i (i = 1,2…14) Arrange and number the points in according to the distance from A, so that the first column of the calibration board is found, and the set is updated P = P-{p _i };

Step2: The point farthest from the straight line l in the updated point set, Calculate the straight line parallel to the straight line with as the point, repeat Step1, and find the second column of the calibration plate, and so on until you find the ninth column of the calibration plate;

Step3: The next two steps are similar to the previous two steps. First, find 14 points on the straight line, sort and number these points according to the distance from point C from far to near, and record as P_i and update the collection P = P-{p _i }, his is the 14th column of the calibration board;

Step4: The point farthest from the straight line in the updated point set, mark as point, use calculate the line parallel to the line for the Repeat Step 3 to find the 13th column of the calibration board, and so on until you find the 8th column of the calibration board.

• Camera calibration experiment

In order to verify the feasibility of the system calibration algorithm in this chapter, this article has photographed the calibration board from multiple angles. Using the method in this chapter, the matching effect of different angles is shown in Figure 9. It can be seen that the camera calibration algorithm used in this article can identify the centers of the calibration plates one by one at different angles.

Figure 9.

Figure 10.