The accuracy (trueness and precision) of Bellus3DARC-7 and an in-vivo analysis of intra and inter-examiner reliability of digital and manual anthropometry

The three-dimensional (3D) analysis of facial morphology is important in many medical and dental disciplines, including orthodontics.¹ A primary orthodontic objective is to create a harmonious soft tissue profile, rather than focussing solely on the teeth.² It is widely accepted that orthodontic treatment of the jaws and teeth can have softtissue effects some of which are desirable and some deleterious. For example, maxillary expansion is believed to be accompanied by an increase in nasal volume due to a widening of the nasal floor, but concomitant widening of the alar base has also been reported.^3,4

There is a desire to better understand and quantify the soft tissue effects of orthodontic and orthopaedic interventions.

An improved understanding of the interrelationship between the teeth, jaws and soft tissues using 3D imaging has the potential to allow for more comprehensive orthodontic diagnosis, treatment planning, treatment acceptance, and an analysis of treatment progress and outcomes.

The era of two dimensional (2D) photographic imaging is swiftly coming to an end and the transition towards 3D facial imaging is becoming commonplace.⁵ Traditionally, soft tissue analysis has been performed using 2D images, lateral cephalograms and callipers for direct anthropometry.^6,7 With the introduction of high accuracy 3D images, these analyses and measurements can potentially be performed with greater precision.⁸ Additionally, there is potential for 3D imaging to provide enough diagnostic information to parallel lateral cephalograms, which could, in time, reduce the need for radiation exposure to patients. One method of 3D stereophotogrammetry is via the use of the Bellus3D ARC-7 system which is a novel multi-camera 3D facial scanning technology that captures commercial grade, 3D, full facial scans. At the time of writing, the authors were unaware of any published research pertaining to the accuracy of the Bellus3D ARC-7 system. Therefore, the primary aim of the present study was to assess the accuracy of the Bellus3D ARC-7 camera by comparing the measurements of soft tissue facial landmarks obtained from Bellus3D ARC-7 scans to the measurements taken manually using Vernier callipers. The secondary aims were to assess intra- and inter-observer accuracy of digital measurements and manual measurements, to identify the most reproducible facial landmark measurements, to identify the difference between the accuracy of measurements of facial landmarks at rest and on smiling and to assess the reproducibility of measurements across multiple scans taken of the same face.

Previous studies have investigated the inter- and intra-examiner reliability of facial measurements taken with digital calipers.⁷ Intra-examiner reliability was found to be high for all measurements while inter-examiner reliability varied, with some landmarks reliable (nasal width at widest nostril and subnasale to upper lip), and others unreliable (base of nose, mouth height, and soft tissue B point to gnathion). This was attributed to the variability in the ease of identification of certain facial landmarks over others.

Although the literature relating to Bellus3D ARC-7 system is sparse, there have been studies validating the Bellus 3D Face app and Bellus Pro which are sister products to the Bellus3D ARC-7.

A study concerning the accuracy of the Bellus 3D Face app was published by Amornvit in 2019.⁸ Measurements taken using multiple 3D facial scanning technologies were compared to ‘gold standard’ measurements which were obtained using Vernier callipers. The measurements were generated from a printed face model and were repeated five times to test intra-observer accuracy. In this study, it was reported that the Bellus Face app was less accurate than other technologies (EinScan Pro 2x Plus), particularly when measuring to a depth of 2mm.⁸

In 2020, Piedra-Cascon et al.⁹ published research on the trueness and precision of the Bellus Face ‘app’ when compared to measurements taken using digital callipers. In this study, the faces of ten participants were measured on two occasions by two observers. The measurements were compared with clinical measurements obtained using digital callipers, on two separate occasions. Significant differences were found between manual and digital inter-landmark measurements in all participants. However, it was concluded that the Bellus Face ‘app’ system produced clinically acceptable outcomes for virtual treatment planning.⁹

Most recently in 2021, Pellitteri et al.¹⁰ reported an in vivo study comparing the Face Hunter facial scanner and the Bellus 3D Dental Pro ‘app’ for facial scanning using digital callipers as the gold standard. A total of four measurements were taken on twenty-five participants. Geomagic X Control software was employed for the superimposition of the scans, to determine the best-fit alignment and to calculate the proportion of overlapping surfaces. There was no statistically significant difference between the accuracy of the Bellus 3D Pro ‘app’ and the Face Hunter scanner. The cheeks were the area associated with the highest average percentage of surface reproducibility. It was concluded that 3D scans of the face are a useful adjunct to the repertoire of diagnostic records.¹⁰

The 3DMD scanner is largely considered to be the gold standard of 3D facial scanners.¹¹ In a study by Zhao et al. in 2017,¹² a 3DMD scan was compared to an industrial laser scanner and identified that the accuracy of obtained 3DMD scans for facial deformities was 0.58 ± 0.11 mm. The 3D accuracy of different facial partitions was inconsistent; the middle face had the best performance. Despite this, the 3DMD scan met the requirements for clinical use.

In 2005, Kau et al.¹³ investigated the accuracy of the Minolta laser scanner. It was found that the reproduction of facial morphology was accurate to within 0.85 mm and concluded that the 3D imaging system (Minolta) is a reliable tool for the study of changes in facial morphology due to treatment and growth.¹³

Kovacs et al.¹⁴ compared measurements from laser scanner data to measurements obtained by manual measurements on a human model. Less than 7% of all results with the scanner method were outside a range of error of 2 mm. Accuracy proved to be sufficient to satisfy requirements for numerous clinical applications.¹⁴

In a systematic review of mobile device facial scanners in 2020, Mai et al.¹⁵ concluded that overall, mobile device-compatible face scanners did not perform as well as professional scanning systems in 3D facial acquisition, but the deviations were within a clinically acceptable range of <1.5 mm. It was also recommended that caution be exercised when interpreting results from studies conducted on inanimate objects such as dummies and models.¹⁵

As a sample of convenience during the COVID-19 lockdowns, four orthodontic registrars at the Sydney Dental Hospital, Sydney, Australia, in 2021 volunteered to be subjects. The sample included three females and one male who presented as adults (age 29–34) with no facial deformities, scars, physical disabilities, previous facial trauma or lip incompetence.

Nineteen anatomical sites were selected on the facial soft tissues of each subject in order to perform various measurements in the three planes of space (Table 1 & Figure 1). These sites were marked directly on the skin with a standard black ballpoint with a 1mm nib (PaperMate InkJoy) pen or on a white, circular 8 mm sticker (Avery Dot Stickers).

Figure 1.

For the anthropometric measurements, the subjects were instructed to hold their breath and remain still in natural head position (NHP) with their eyes open, while distances between landmarks were measured using digital Vernier callipers (Orthopli Corp, USA). Sixteen measurements were recorded to two decimal points. In an effort to minimise intra-examiner error, each subject was measured twice in a resting pose (the mean average score used) and once in a smiling pose. The sequence of sixteen measurements was randomised for each subject to reduce fatigue bias. The digital calliper was recalibrated to zero between all measurements. All measurements were duplicated for each subject by a second examiner, to assess inter-examiner reliability.

3D image capture of the subject’s face using the Bellus3D ARC-7 camera was performed (Figure 2) in a controlled space with standardised lighting conditions on the same day as the standard anthropometric measurements. The subjects were seated in a preadjusted chair and head posture was standardised in NHP as determined by instructing the subjects to look level towards the horizon. The Bellus3D ARC-7 camera was then adjusted to this height. By the use of a plastic ruler measuring to the subnasale region, subjects were positioned 30.5 cm from the central Bellus camera and this distance was confirmed as satisfactory by the Bellus3D ARC-7 system which produces a green circle when the subject is at the correct distance from the central camera. Five facial scans in a resting pose and five additional scans in a smiling pose were captured for each of the subjects. The scans were transferred onto secure files on the chief investigator’s password protected computer in the form of .obj files of approximately 165 MB in size.

Figure 2.

Via randomisation provided by (www.random.org), ‘At rest’ scan number 2 and 3 of the five scans taken at rest and ‘Smiling’ scan number 3 of the five scans taken on smiling, were selected for all subjects for analysis. Stereophotogrammetry of the digital images was performed using a 3D imaging software (3D builder, Microsoft Corporation, Seattle, USA) and anthropometric measurements conducted on the facial soft tissues were replicated on the 3D model. ‘At rest’ scan number 2 was measured twice by the examiner to assess intra-examiner reliability.

All stereophotogrammetric measurements of the 3D images were performed twice by two different examiners to assess inter-examiner reliability.

For each subject, the anthropometric and stereophotogrammetric data were collected and recorded independently to minimise recording bias. The results were collated on Microsoft Excel for analysis.

In collating the data, one landmark (R Upper Ear-Middle Ear) was excluded because the measurement point was covered by hair and could not be located on the digital model.

Precision of the digital model

The mean difference in inter-landmark measurements of two consecutive digital models taken of the same sample is shown in Table II. The left (L) Gonion-Commissure was the only measurement that showed deviation greater than 1.00 mm (Figure 3).

Figure 3.

Table II.

Mean difference in measurements of inter-landmark measurements taken on two consecutive digital models of the same sample

Landmark 1		Landmark 2		Mean Difference (mm)
Ref		Glabella		0.25
Glabella		Nose Tip		0.19
Subnasale		Pogonion		0.54
R Cheek		L Cheek		0.46
R Alar		L Alar		0.38
R Commissure		L Commissure		0.87
R Upper Ear		R Middle Ear		N/A
R Middle Ear		R Lower Ear		0.57
R Cheek		R Gonion		0.53
R Cheek		R Middle Ear		0.69
R Gonion		R Commissure		0.51
L Upper Ear		L Middle Ear		0.47
L Middle Ear		L Lower Ear		0.37
L Cheek		L Gonion		0.23
L Cheek		L Middle Ear		0.27
L Gonion		L Commissure		1.11
Colour Legend:	0 - 0.5mm	0.5 - 1.0mm	1.0 - 1.5mm	1.5 - 2.0mm	2.0+mm

Inter-examiner reproducibility

The inter-examiner reproducibility of each of the inter-landmark measurements was analysed as the mean difference of the pooled measurements from all the examiners and models. When measured using callipers, the Cheek-Gonion (Right and Left) and R-L Cheek measurements demonstrated mean differences greater than 1.00 mm between different examiners (Table VII.). When measured digitally, the L Gonion-Commissure was the only inter-landmark measurement that deviated greater than 1.00 mm (Table VIII). The R Upper Ear-Middle Ear was excluded because the landmarks were covered and could not be located. All remaining digital measurements had a mean difference of less than 0.5 mm (Figures 4 and 5).

Table VII.

The mean difference in pooled manual measurements of inter-landmark measurements taken by all examiners

Landmark 1		Landmark 2		Mean Difference (mm)
Ref		Glabella		0.35
Glabella		Nose Tip		0.64
Subnasale		Pogonion		0.90
R Cheek		L Cheek		1.14
R Alar		L Alar		0.65
R Commissure		L Commissure		0.99
R Upper Ear		R Middle Ear		0.70
R Middle Ear		R Lower Ear		0.28
R Cheek		R Gonion		1.28
R Cheek		R Middle Ear		0.80
R Gonion		R Commissure		0.69
L Upper Ear		L Middle Ear		0.72
L Middle Ear		L Lower Ear		0.47
L Cheek		L Gonion		1.38
L Cheek		L Middle Ear		0.62
L Gonion		L Commissure		0.26
Colour Legend:	0 to 0.5 mm	0.5 to 1.0 mm	1.0 to 1.5 mm	1.5 to 2.0 mm	2.0+mm

Table VIII.

The mean difference in pooled digital measurements of inter-landmark measurements taken by all examiners

Landmark 1		Landmark 2		Mean difference (mm)
Ref		Glabella		0.25
Glabella		Nose Tip		0.19
Subnasale		Pogonion		0.23
R Cheek		L Cheek		0.11
R Alar		L Alar		0.09
R Commissure		L Commissure		0.31
R Upper Ear		R Middle Ear		N/A
R Middle Ear		R Lower Ear		0.28
R Cheek		R Gonion		0.33
R Cheek		R Middle Ear		0.23
R Gonion		R Commissure		0.29
L Upper Ear		L Middle Ear		0.23
L Middle Ear		L Lower Ear		0.41
L Cheek		L Gonion		0.26
L Cheek		L Middle Ear		0.32
L Gonion		L Commissure		1.16
Colour Legend:	0 to 0.5mm	0.5 to 1.0mm	1.0 to 1.5mm	1.5 to 2.0mm	2.0+mm

Figure 4.

Figure 5.

Comparison of manual and digital measurements

Table IX shows the mean difference between manual and digital inter-landmark measurements taken at rest. Just over half of the digital measurements were within 1.0 mm of the manual measurements. All digital measurements deviated less than 2.00 mm from manual measurements, with the exception of L Gonion-Commissure (Figure 6).

Figure 6.

Table IX.

The mean difference between digital and manual inter-landmark measurements on models (at rest)

Landmark 1		Landmark 2		Mean difference (mm)
L Upper Ear		L Middle Ear		0.27
R Middle Ear		R Lower Ear		0.52
R Cheek		R Middle Ear		0.56
Glabella		Nose Tip		0.61
L Middle Ear		L Lower Ear		0.61
Ref		Glabella		0.71
L Cheek		L Middle Ear		0.78
R Alar		L Alar		0.94
R Cheek		L Cheek		1.10
Subnasale		Pogonion		1.11
R Gonion		R Commissure		1.13
R Commissure		L Commissure		1.52
L Cheek		L Gonion		1.63
R Cheek		R Gonion		1.76
L Gonion		L Commissure		2.12
R Upper Ear		R Middle Ear		N/A
Colour Legend:	0 to 0.5 mm	0.5 to 1.0 mm	1.0 to 1.5 mm	1.5 to 2.0 mm	2.0+mm

The mean difference between manual and digital inter-landmark measurements taken on samples during smiling are shown in Table X. Inter-landmark measurements that matched most closely between digital and manual measurements, in both rest and smiling models, included Ref-Glab, Glab-Nose tip, Upper-Middle Ear (Left), Lower-Middle Ears (Right and Left) and Cheek-Middle Ears (Right and Left) (Figure 7).

Figure 7.

Table X.

The mean difference between digital and manual inter-landmark measurements on models (during smiling)

Landmark 1		Landmark 2		Mean difference (mm)
Ref		Glabella		0.39
L Upper Ear		L Middle Ear		0.31
R Middle Ear		R Lower Ear		0.46
Glabella		Nose Tip		0.57
L Middle Ear		L Lower Ear		0.66
R Cheek		R Middle Ear		0.79
L Cheek		L Middle Ear		0.95
R Cheek		R Gonion		1.24
L Cheek		L Gonion		1.33
R Alar		L Alar		1.44
R Cheek		L Cheek		1.68
L Gonion		L Commissure		1.85
R Commissure		L Commissure		1.88
Subnasale		Pogonion		2.20
R Gonion		R Commissure		2.28
R Upper Ear		R Middle Ear		N/A
Colour Legend:	0 to 0.5mm	0.5 to 1.0mm	1.0 to 1.5mm	1.5 to 2.0mm	2.0+mm

3D facial scanners are becoming increasingly popular and play an important role in the assessment and treatment planning phases of orthodontics. New facial cameras are constantly emerging and their rendered images can influence treatment plans and decisions. It is therefore important for these cameras to capture accurate images which are a true reflection of reality. This enables precise measurements to be performed with consistency.

The Bellus3D ARC-7 system captures 3D high resolution images using up to 7 cameras. It is marketed as a “revolutionary multi-camera 3D face scanning solution that captures commercial grade, full 3D face scans in less than three seconds”. There is currently no literature reporting on the accuracy of the Bellus3D ARC-7 system.

Manual anthropometry is currently rivalled by 3D images and digital technology. In the current study, manual and digital anthropometry were conducted using ink demarcations on facial landmarks of the subjects. A challenge of this method is the reproducibility of the results, both within and between examiners. In the present study, each 3D facial scan, both smiling and at rest, was digitally measured twice by the same examiner. Manual measurements were performed once with the subject smiling and twice with the subject at rest. The intra-examiner results revealed that 40% of the manual measurements using callipers were reproducible within ±0.5 mm, and 72% were reproducible within ±1 mm. The intra-examiner digital measurements proved to have increased reproducibility as 88% of measurements were reproducible within ±0.5 mm and 98% within ±1 mm. The lower reproducibility result associated with the manual measurements may be attributed to factors related to the technical complexity of calliper use, soft tissue compressibility, operator fatigue, the variability of the model and examiner positioning, and the unreliability of the facial landmark stickers. Nevertheless, measurements within a 2 mm range are currently considered an acceptable range of validity based on previous studies.^16,17 Ninety-three percent of manual measurements using callipers were within a ±2 mm range and 100% of digital measurements on the 3D scans were within a ±2 mm range. Therefore, if 2 mm is used as an acceptable range, both the traditional manual method and digital method can be considered as clinically reproducible intra-examiner measurements.

Inter-examiner manual measurements showed greater heterogeneity compared to inter-examiner digital measurements, however, still within the acceptable 2 mm range. The mean difference in manual measurements between examiners ranged from 0.26 mm to 1.38 mm. The mean difference in digital measurements between examiners was much narrower and ranged from 0.11 mm to 0.41 mm, except for the measurement from L-Gonion to L-commissure (1.16 mm) which could have been a transcription error. It was observed that larger distances between facial landmarks correlated with an increased incidence of inaccuracies. This demonstrates the variability of measurements between different examiners which is influenced by the method and equipment utilised, and also the distance between landmarks. The mean difference in digital measurements was significantly less for the majority of inter-landmark distances, hence digital measurements had more reproducibility compared to manual measurements. However, both methods still measured within the acceptable 2 mm range and proved to be reproducible between examiners.

Digital measurements on 3D facial scans were compared to manual measurements on the face to validate facial imaging as an accurate diagnostic and treatment planning tool in orthodontics. The mean differences between digital and manual measurements varied between 0.27 to 2.12 mm (at rest) and 0.27 to 2.29 mm (smiling) across facial landmarks. 79.5% of measurement deviations (at rest) fell within the ±1.0 mm range and 95.1% were within the ±2.0 mm range. Six of 122 measurements (4.9%) at rest had a measurement deviation of >2.0 mm between digital and manual measurements (Bell Curve Figure 8). Landmarks which produced the most varied measurements were:

Cheek to cheek

Figure 8.

These landmarks were of larger distances, around facial curvatures or in the lower face where muscles are more mobile. Reproducible measurements were mainly located around the forehead and ears where distances were shorter and the soft tissues had minimal mobility:

Reference to Glabella

Although Pellitteri¹⁰ reported that facial surface scans were accurate and comparable to manual methods, some of their results were contradictory to the current study. The Pellitteri scans showed that the cheek area measurements fell within the highly reproducible range (about 60%), followed by the chin and the tip of the nose. The forehead area was also described as having poor reproducibility, with more than 20% of values out of tolerance; however this was not the case in the present study.¹⁰

The facial scans produced by the Bellus3D ARC-7 system presented some distortion, particularly around the lateral aspects of the face (Figure 9, Figure 10). Distortion of the stickers used to mark landmarks were visible on some of the scans. Surprisingly, image distortion did not appear to significantly affect the accuracy or variability of the measurements of the lateral cheek or ear. This is likely due to the stability of the landmarks which have little mobility compared to other areas of the face. The ear cartilage is also less compressible which minimises distortion of the tissues with calliper measurements. The distance between the landmarks was also relatively short which, as previously discussed, lends itself towards improved measurement accuracy.

Figure 9.

Figure 10.

The digital measurements displayed in Appendix 1 and 2 were not consistently higher or lower compared with calliper measurements. The digital measurements had a tendency towards overestimation. This could be caused by a parallax error since the face orientation can be adjusted on the 3D imaging viewer software. It can be concluded that the 3D image captured by the Bellus camera is an acceptable representation of the model as no obvious magnification of the image was identified.

Although multiple scans were performed, a limitation of the present study was the small sample size of four subjects who were also the examiners. The inclusion of multiple examiners can introduce errors due to the variability in measuring techniques, although effort was made to standardise this process. Of note, the examiners did not receive any formal training in direct anthropometry. At the time of scan capture, a standardised distance of 30.5 cm between the model and camera was required. However, the maintenance of the positioning could not be guaranteed because the subject’s head was not immobilised at this camera distance.

The images captured by the Bellus3D ARC-7 system are an acceptable representation of the subject’s extra-oral soft tissues, facial form and size, which can have various applications in orthodontic patient management. It can be useful during the initial stages of diagnosis, treatment planning, and surgical treatment simulation. Treatment discussions with the patient or other clinicians is made easier and can be utilised to improve case acceptance. Throughout treatment, digital scans may be used to assess treatment progress and analyse post-treatment outcomes. Growth monitoring and assessment of facial changes is also possible with sequential scans. In the future, there is potential for facial scans to be integrated with CT or CBCT scans leading to the possibilities for future research on facial analysis in a non-invasive way.

Manual measurements of facial landmarks using Vernier callipers were less reproducible than digital measurements made using the Bellus3D ARC-7 system. The digital method produced more consistent intra- and inter-examiner measurements. Nevertheless, intra- and inter-examiner reliability and reproducibility tests proved to be acceptable within the 2 mm range for both manual and digital measurements. There was a high consistency of measurements taken across multiple digital scans of the same subject, demonstrating and confirming the reproducibility of the system.

The mean differences between calliper and digital measurements varied between 0.27 to 2.12 mm (rest) and 0.27 to 2.29 mm (smiling). 4.9% of the measurements differed by more than 2 mm between the manual and digital techniques. The remaining measurements (95.1%) were within the acceptable 2 mm range. More stable measurements were located on less mobile parts of the face and were of a shorter distance, such as the ears and forehead. Landmarks and measurements around the lower half of the face, especially around the mouth, were the most varied using direct anthropometry and therefore demonstrated greater deviation from measurements taken using digital methods.

The 3D image was not shown to be magnified and was an acceptable representation of the subject. Distortion of landmarks occasionally occurred, particularly around the ears, lips and hair which were difficult to capture accurately. Despite this, 3D facial scans captured by the Bellus3D ARC-7 system proved to be an accurate and true image of the face from which reproducible measurements may be made within and between and examiners.