Digital image processing technique to measure the range of motion of the elbow

This study was approved by the Institutional Review Board of the Faculty of Medicine, Chulalongkorn University, Bangkok, Thailand (Certificate of Approval No. 198/2019). Participants were recruited from students and staff in the university by poster announcement. Inclusion criteria included all volunteer participants aged over 18 years who were able to lift their shoulder perpendicular to the floor and hold it still in an elbow extension and flexion position. Exclusion criteria included those who had deformity of arm and forearm or pain and discomfort at the elbow. Participants were informed about study protocol and risks involved in this study. The written informed consent was provided. Sample size was determined by calculating two-dependent mean sample sizes. Ten degree of elbow motion arc measurement error was considered as minimal clinically important difference (MCID) and used as summative difference. Standard deviation was obtained from the difference between photographic and goniometric measure ments [7]. A total of 14 elbows were required.

A digital goniometer and digital inclinometer were used as reference measuring devices to compare ROM measurements with those from DIPT. All measurements and capturing photographs were conducted on the same day (April 20, 2019) in the orthopedic operating room of King Chulalongkorn Memorial Hospital. Two trained physiotherapists with 2 years’ experience were briefed about the measurement techniques and they practiced the measurements on each other. Both examiners performed goniometer, inclinometer, and ROM measurements on the bilateral elbows of participants. Two research assistants recorded the measurement values on record forms.

Digital goniometer

A digital protractor goniometer with ±0.5° precision (Mediguage) was used. The goniometer was centered on the lateral epicondyle. The proximal part of goniometer pointed at the greater tuberosity of humerus and the distal part pointed at the middle portion of wrist [2]. The examiner measured the flexion and extension position of elbow three times for each side.

Digital inclinometer

A baseline digital inclinometer (Fabrication Enterprises) was calibrated. The measurement technique for the inclinometer was modified from the American Medical Association (AMA) recommendation [9]. The participant laid supine on a bed and both elbows hung beyond the edge of the bed. To measure the extension angle, the inclinometer was aligned on the long axis of forearm and set to zero. The participant was then asked to extend his or her elbow and the measurement value was recorded. To measure the flexion angle, the participant fully flexed his or her elbow while the examiner aligned the inclinometer with the forearm, read the flexion angle, and then repeated the same protocol two more times for extension and flexion. Three measurement values for the same limb and same position should be within 5° or 10% of the mean.

Smartphone photography

The participant stood as close as possible in front of a blue screen to control the horizontal plane of the arm and then performed lateral abduction of the shoulder to 90° perpendicular to the floor. This position is considered as practical for a patient to take a photograph of his or her elbow at home and the monotonous color contrast background should help reduce image processing error. The upper extremity was exposed from shoulder to hand. A photographer took elbow images in with the arm fully extended and fully flexed three times for each position (Figure 1). The smartphone camera was at the same level of the elbow joint while taking a shot. Participant was told to drop his or her arm to a relaxed position between each photo shot and then raised it again so that the next image could be acquired. All images were taken using a iPhone 6 (Apple) with 8-megapixel rear camera (3,264 × 2,448 pixels, 72 dpi).

Figure 1

Digital image processing technique

The first step is “line detection,” which is the protocol for finding all possible lines in an image. All upper extremity images were cropped at below wrist level in the distal part and at deltoid muscle insertion in the proximal part. The horizontal and vertical blue areas were deleted from edges to eliminate the blue screen background as much as possible (Figure 2A). A median filter was applied to reduce noise from the camera and reduce light by converting the colored space from red, green, and blue (RGB) to hue saturation value (HSV) to create a mask with color range (100,0,0) and (180,255,255) in the HSV color space (Figure 2B). A mask was then applied to find the contour of the skin and clean the noise with a median blur filter (Figure 2C). Detection of lines was based on a Canny and Hough transformation (Figures 3A and 4A) [10, 11]. To improve accuracy, outlier detection was used to eliminate unrelated lines.

Figure 2

Figure 3

Figure 4

After the line detection process, the next step is “angle calculation.” The algorithm for flexion and extension angles is similar. They varied depending on sides. For left flexion, right flexion, left extension, and right extension, the variation are the base lines for detecting upper and lower arms. For both flexion and extension, four base lines (upper arm, lower arm, upper forearm, and lower forearm) are located differently. The protocols for calculation of extension and flexion angles are described as follows.

Extension angle

(1)

Two scan lines, such as a left scan line and a right scan line, were determined. The first scan line was located in one-third distance of the dominate edge (right edge for the left hand and vice versa). The second scan line was a quarter from another edge (Figure 3A).

(2)

For both scan lines, we found the cutting points for classifying lines into two groups on each side. The cutting points were determined by the middle point of the skin (calculated from the mask).

(3)

Lines were partitioned into four groups: top left lines, top right lines, lower left lines, and lower right lines. Between the left edge and the middle point, the intersection of each line and the scan line was used to determine the upper left lines and the lower left lines. A similar method was applied between the right edge and the middle point to determine the upper right lines and the lower right lines.

(4)

To calculate the angle, the average slopes between the upper lines and the lower lines were calculated. The angle was calculated from the difference between the two slopes (Figure 3B).

Flexion angle

(1)

A scan line was determined by using a vertical line from 10% of the edge. Either the left edge or right edge was used based on the side of the elbow.

(2)

The scan line was scanned to find the cutting points for classifying four lines. The top point was determined by the middle point of the forearm. The lower point was determined by the middle point of the arm. The middle point between the empty spaces in the middle was used to partition between the forearm and the arm (dots in Figure 4A).

(3)

For each line, we determined whether the line was a part for the upper or lower line by calculating the intersection at the scan line. The intersection point was then partitioned into 4 groups based on the detection points in the second step: upper top lines, lower top lines, upper low lines, and lower low lines (colored lines in Figure 4A).

(4)

For each group of lines, we found and averaged the slope. There were slopes.

(5)

We used a slope from the upper top line as the top reference line. This line is likely to be aligned with the ulnar bone. We used an average between the 2 slopes from the lower top lines and lower low lines as a base line. The angle was calculated from the 2 slopes, such as top reference line and base line, using an arctan function (Figure 4B).

To validate our algorithm, we implemented our design using Python version 3.6. The imaging processing library is OpenCV [OpenCV] version 3.3. The outlier detection was based on local and outlier factor found in the Scikit-learn library version 0.19 [12].

Statistical analyses

Intra- and inter-rater reliability of all methods was computed using an intraclass correlation coefficient (ICC). The model for intrarater reliability of goniometer, inclinometer, and photographic image analysis was “two-way mixed” and “absolute agreement.” Inter-rater reliability of goniometer and inclinometer between two physiotherapists’ analytical models was “two-way random” and “absolute agreement.” Excellent agreement was determined by an ICC value >0.9, good agreement 0.75–0.90, and moderate agreement 0.50–0.90 [13]. Reliability test was conducted using IBM SPSS for Windows version 22.

The average of flexion and extension scores from both examiners and DIPT were used in the analysis. Total ROM value was the flexion angle minus the extension angle. Minimal clinical significance difference for elbow ROM was considered as 10°. Kolmogorov–Smirnov and Shapiro–Wilk tests were used to verify if variables were normally distributed. A paired Student's t test and Wilcoxon-signed rank test were used to detect systematic bias between all measurement techniques. Bias and possible range of difference between methods were illustrated using Bland–Altman plot and limits of agreement (LOA) analysis. Bland–Altman analysis was conducted in R (R Core Team) using the “blandr” package [14, 15]. DIPT measurement values that were <10° of difference were compared with the other 2 reference methods calculated in percentage.

Thirty healthy volunteer participants joined this study. After inspecting all images, two participants were removed from final analysis because their elbows were not fully flexed. Ultimately, we used data from a total 56 elbows from 15 male and 13 female participants. The average age was 20.6 years (range: 19–31 years). Mean weight, height, and body mass index were 58.9 kg (range: 39–110 kg), 165.3 cm (range: 150–189 cm), 21.4 kg/cm² (range: 16.7–34.0 kg/cm²).

Intrarater ICC of goniometric and inclinometer in flexion and extension position showed excellent agreement between 0.953 and 0.994. Intrarater ICC of DIPT was 0.943 in extension and 0.886 in flexion. There was moderate-to-good inter-rater reliability of extension (E) and flexion (F) angle ICC between the two examiners, 0.862 (E) and 0.738 (F) for goniometer and 0.882 (E) and 0.784 (F) for inclinometer.

All variable groups, except extension position of inclinometer group (P = 0.026), were normally distributed. Mean extension and flexion angles as measured by DIPT were significantly greater than those measured by goniometry and inclinometry (P < 0.05), but total ROM was comparable with both reference methods (DIPT vs goniometer P = 0.322, DIPT vs inclinometer P = 0.534) (Table 1).

Normality of score differences in image analysis and the goniometer and inclinometer methods were assessed using a Shapiro–Wilk test. All methods demonstrated a normal distribution pattern (P = 0.06–0.39).

Bland–Altman analysis showed extension and flexion angle bias of the DIPT–goniometer 4.51 (95% CI 3.14–5.88), 5.46 (95% CI 4.12–6.80) and DIPT–inclinometer 4.61 (95% CI 3.25–5.96), 3.98 (95% CI 2.45–5.50). Total ROM mean difference of DIPT–goniometer and DIPT–inclinometer was 0.94 (–0.95 to 2.84) and –0.63 (–2.64 to 1.38). Absolute maximal error of flexion and extension angles was 9.81°–11.17° and total ROM angle was 13.84°–15.99°. There were 80.4%–91.1% of DIPT values that were <10° of MCID compared with goniometer and 83.9–87.5% compared with inclinometer (Table 2 and Figures 5 and 6).

Figure 5

Figure 6

There are several advantages of using photography-based ROM measurement, especially in the era of telemedicine. Innovative methods to collect data remotely from a patient have been popularized in recent years. Physicians can assess their patients’ function and make suggestion via Internet portal without having patients come to the clinic. Photography- and video-based ROM measurement method required human observers who need proper training to achieve high accuracy (7). Most photographic–goniometric methods use bony or alternative landmarks for drawing two reference lines that may be difficult to locate in some cases (1, 4, 5, 7). The present study aimed to solve these observer-related problems using DIPT.

The present study has some limitations. First, DIPT protocol assesses only the active ROM. An examiner is needed to perform passive ROM measurement. Second, some participants may not produce their best effort, do not understand instructions clearly, and do not extend or flex their elbow fully. We needed to exclude four images from the analysis. Third, the extension rod as recommended by AMA was not available for digital inclinometer used in the current study.

DIPT measurement of flexion and extension had higher bias around 4°–5° than in the other methods. There are various explanations for this deviation. First is the difference in vertex location, and goniometer and inclinometer measurements begin with localizing lateral epicondyle as vertex of angle and then projecting its arm to distal and proximal bony landmark. DIPT is different from reference methods, it uses extremity contour to create proximal arm and distal forearm line, and the vertex is intersected between these lines. Second, the dorsal surface of the forearm is thin and close to ulnar shaft alignment which affects distal reference line when calculating angle using DIPT. Also, vertex of goniometer and inclinometer angles lies anteriorly compared with ulnar bone shaft line. This is demonstrated from geometric illustration obtained using fluoroscopic images of the elbow which use similar DIPT protocol for drawing angle alignment (Figures 7A and 7B). Another possible cause can be found from images review, and some participants minimally flexed instead of fully extended their elbows in extension image, probably because they used biceps muscle function to control shoulder joint in the abduction plane.

Figure 7

Total ROM had comparable result with reference methods, but had an absolute error >10° margin of MCID for elbow joint. Total ROM is the calculated value from flexion angle minus extension angle, so the error is combination between these two values. Twenty from total 116 DIPT measurements, 11 compared with goniometer and 9 compared with inclinometer, had difference >10°. These can be divided into 3 groups: group 1 denotes lower extension angle and higher flexion angle; group 2 denotes higher extension angle and lower flexion angle; and group 3 denotes higher extension and flexion angles. There were 8, 7, and 5 measurements in group 1, group 2, and group 3, respectively. Of note, 15 of 20 measurements had higher value and more effect than lower value, and other 5 measurements had nearly equal effect between higher and lower values. This can be speculated that significant difference of total ROM between DIPT measurement and reference methods occurred because DIPT measurement tended to have higher value of either flexion or extension angle than goniometer and inclinometer measurements.

There are several studies using photograph for elbow arc of motion measurement with variety of methods (Table 3). Image capturing devices in literatures were either digital camera [4, 7, 8], or smartphone [1, 5]. Most reports except those reported by Russo et al. use lateral side of arm and forearm for measurement. Healthy volunteers were recruited in most studies, but Blonna et al. tested with patients and Russo et al. used cadaveric elbows. The accuracy of photography-based elbow ROM measurement among studies varied and also depended on observer's experience [7]. The mean differences of current study were comparable with previous literature; however, the error margins were higher.

There are some implementation problems of DIPT that should be concerned. First, the initial position for photographic images was 90° lateral abduction of shoulder, which was different from other standard methods, in which elbow joint lied beside torso in anatomical position. The reasons for modification are outline detection function that requires body lie on the monotonous background. For this reason, the current method cannot be used in some patients, if they have problems such as shoulder joint stiffness or muscle weakness, thus they cannot lift their elbow against the background. Second, the compliance of patient to obtain valid images is very important. It is important to follow proper image capturing protocol to prevent photographic error such as incorrect projection. Third, in case of loss in the normal contour of arm or forearm from injury, morbid obesity, or other diseases in patients, this method may not give an accurate result. Finally, image should be cropped by the observer in the current protocol. Developing algorithm to detect anatomy and position of arm and forearm may help alleviate this burden.

This research developed an innovative method for measuring ROM from photograph of elbow. Elbow ROM measurement from current DIPT protocol had comparable result with goniometer and inclinometer, but it can be different from other two methods up to 15.99°. Further investigations and protocol adjustment are needed to increase the accuracy of the image analytic technique.