The use of wearable technology to measure and support abilities, disabilities and functional skills in autistic youth: a scoping review

A range of head-mounted technologies were described, including glasses or devices worn in the form of a headband or fitted hat/cap. These WTs were used to capture data such as head orientation, gaze patterns, pulse and electroencephalography (EEG).

Smart or assistive reality glasses were discussed by 10 studies (Table 1). These glasses typically had an outward facing camera capturing the faces of others, and an inward facing tracker monitoring the wearer’s eye gaze patterns and were capable of providing youth with real-time feedback on their social function through providing audio or visual feedback. Several systems utilizing smart glass technology were presented as a means to support and improve social communication and interaction in autistic youth. The Brain Power System (31), Brain Power Autism System (32-34) and the Empowered Brain (formerly Brain Power Autism System) (32) used smart glasses as a social communication aid for autistic individuals, providing personalized coaching on gaze behavior and emotion recognition through gamified interaction. The SuperPower Glass system (35, 36), was also designed to support emotion recognition in autistic children through using machine learning software and automatic facial expression recognition capabilities. Similarly, Washington et al. (37) describe a system using automatic facial emotion recognition to support emotion recognition in autistic children, finding that autistic children tended to choose cues provided by the google glasses over their own intuition. While these various smart glass-based systems provide some evidence suggesting that they may improve social engagement (31), eye contact (31, 37), non-verbal communication (31), and emotion recognition (35, 38) in autistic individuals, and may reduce attention deficit hyperactivity disorder symptoms (31, 39), and autistic-like traits (35), these results are inconsistently observed, and are insufficient to draw conclusions regarding the effectiveness of these systems.

One study utilized the onboard microphone of the Google Glass and Google Speech Recognition software to develop the Holli application (40). This application ‘listened’ to conversations and generating conversation prompts and responses visually displayed through the glasses (40). Though preliminary and limited, this system may be useable and effective for supporting communication among autistic children.

Overall, studies investigating smart glasses suggested that they were generally tolerated by autistic youth (31-35), and were perceived as useable (31, 32, 35, 37, 40) and enjoyable (31, 33, 37, 40). Some studies, however, reported that smart glasses may become uncomfortable if worn for long periods of time, or become too warm (34, 37). When examining the potential for negative effects, evidence was found suggesting that a small number of participants experienced mild adverse effects including dizziness, nose-bridge discomfort and eye strain (34).

Other head-mounted technologies capable of measuring eye gaze were discussed (41-43) such as ‘HATCAM’, a headband or hat with an attached camera and mirror, used to examine visual attention to a robotic face during adaptive therapy aimed at improving social communication in autistic children (43). A similar device, the ‘WearCam’, using two cameras and a mirror attached to an elastic headband, provided researchers with a means to examine gaze behavior during complex social interaction (41, 42).

While not tracking eyes directly, a similar concept was employed by Spiel et al. (44), who discussed the ‘ThinkM’, a wearable headset with a pulse sensor and JPEG camera located at the participant’s eye and recorded real-time images every ten seconds. This device was developed in collaboration with an autistic child to assist in reviewing arousal and, interpreting and reviewing social situations after-the-fact, supporting social function.

Head-mounted devices to explore and quantify face-contact or orientation to another person were explored in two studies (45, 46). A prototype head-mounted device with OpenCV Face detection to track head position and determine face contact showed promise in detecting face contact during conversations in a fixed, stable environment, however, significant difficulties were noted in regard to the system’s ability to accurately detect face contact and adjust to environmental factors (such as lighting) (45). Another study used a headband device, ‘FaceLooks’, using infrared communication to quantify mutual face gaze behavior (46). This WT contained infrared lights that changed color depending on whether face-to-face behavior was achieved. This feedback was associated with increased face-to-face orientation, with usability investigation suggesting that participants tolerated and enjoyed wearing the device for short periods of time, indicating that the device may prove useful in intervention and investigation seeking to improve social interaction. It must, however, be noted, that a limitation of the ‘FaceLooks’ device was that all participants were required to wear the headband for the device to function (46). Finally, two studies aimed to use wireless EEG recording devices to examine and detect neurophysiological activities unobtrusively. In one study it was shown to be effective in examining engagement in children during play-based imitation tasks, potentially providing avenues for assessing neural activity during intervention or treatment (47), however, in a later study, difficulties in wireless syncing and obtaining adequate impedance levels precluded its use in the study (30).

Body-worn technologies included sensors placed on the body, such as accelerometers, electromyography sensors and chest belts or sensors. Six articles (Table 1) investigated the use of accelerometers to measure and classify stereotypical motor movements such as body rocking and flapping, with two studies also investigating accelerometers to detect self-injurious behavior (48, 49). Accelerometers were typically worn on the hands and torso, however, one study found evidence to suggest that movements such as body rocking and hand flapping may be adequately detected by a single sensor placed on the back (50). Accelerometers to detect stereotypical movement or self-injurious behavior was observed to be generally accurate (48-52), however, significant variability in individual behavior was proposed to negatively influence classification performance, with individualized and adaptive classification of stereotypical movement and self-injurious behavior still required (53). Electromyography (EMG) sensors placed on the face were discussed as a means to measure smile behavior in autistic children in two studies (54, 55). One study found that the use of facial EMG sensors during animal-assisted therapy were effective in quantifying children’s emotional experiences to the therapy (54). In a later study (55), it was shown that EMG sensors placed on the face were capable of reliably detecting smile behavior in autistic children during play with a robot.

Wearable sensors to record ECG signals, to provide markers for ANS response were used by four studies (30, 47, 56, 57). Feasibility testing suggested that the use of chest-belts to measure ECG in semi-naturalistic paradigms and clinical is effective, and may enable real-time tracking of engagement (47, 56) and autonomic nervous system activity (30, 57). Another chest-worn sensor was the ‘ProCom’, a proximity sensor developed using parallel design with autistic children and adults to support awareness of interpersonal space. Despite some limitations, including children’s tolerability of the device and comfort, the device effectively measured proximity, was useable in everyday life, and resulted in autistic children modifying their proximity to others appropriately (58).

Accessory and clothing based technology included wrist-worn devices (14, 30, 45, 59), sensorised shirts, (43, 60) and technology worn around the neck (61).

Wrist-worn devices were examined in four studies (Table 1). These devices measured physiological signals including skin conductance (45), touch (59), and heart rate (14). One study also used the AMI MotionLogger Sleep Watch to measure sleep onset, quality, and duration (30), while this study also aimed to use a child daytime sensor measure electrodermal activity (EDA), actigraphy, and skin temperature, this daytime sensor was discontinued prior to the study with EDA instead measured using dry electrodes in a clinical setting.

In addition to the head-mounted wearable camera device, Lee et al. (45) reported the use of a bracelet to measure skin conductance levels to provide an index of arousal during social and physical engagement. This study showed that skin conductance levels among autistic participants varied in accordance with the conversation topic and whether that topic was of interest, indicating that skin conductance measured through a wrist device may provide a means to examine physiological information during social interaction. An ‘LG Watch Urbane’ smartwatch was similarly used to enable autistic children to self-regulate their emotions (14). Through measuring heart rate, the smartwatch was capable of detecting anger or distress in real time, with self-regulation strategies developed by parents and teachers displayed through the smartwatch. Observations with two children over nine days showed this device was used and enjoyed by the children, with preliminary evidence suggesting that the device may assist children to self-regulate emotion in the classroom. The wrist device described by Suzuki et al. (59) was developed as a means to measure and encourage physical contact or touch in autistic children. The ‘EnhancedTouch’ was capable of quantifying touch events between two individuals, including frequency and duration of touch and provided visual feedback through emitting light when touch events occurred. While this device was capable of measuring only hand-to-hand touch, it was found to be effective in measuring touch events, was accepted by children, and increased instances of touch when visual feedback was provided (59).

Two studies discussed the use of sensorised shirts as a wearable platform to assess physiological and/or electrocardiographic (ECG) signals. A ‘Smart Clothe’ which contained ECG sensors in the wrist cuffs of a long-sleeved garment worn by children was shown to provide a reliable means for parents and therapists to monitor mental stress during therapy in children (60). A similar wearable garment with sensors woven into the fabric was capable of capturing a range of physiological activities including heart rate, heart rate variability, respiratory rate, skin temperature, and skin conductance during the course of therapy using robots (43).

One study, through consultation with parents of autistic children, used two devices, the ‘SenseCam’ a wearable digital camera worn around the neck and a modified iPod Touch with the Lifelapse application, also worn around the next, were used to capture still images from the wearer’s perspective. The ‘SenseCam’ automatically captured images every one minute, increasing in frequency if changes were detected, while the modified iPod Touch automatically captured images every 30 seconds. These devices both improved understanding of their child’s experiences and needs, and enabled parents and autistic children to review interactions. Concerns were however raised by participants regarding the appearance of the device, which was suggested to be lacking in social accessibility and parents reported difficulty accessing data collected by the devices (61).

The physiological and behavioral functions measured by the WTs, as well as their functional targets, were linked to the body function domain. Six out of the eight body function chapters were represented, including mental functions (b1), sensory functions and pain (b2), speech and voice functions (b3), functions of the cardiovascular, hematological, immunological and respiratory systems (b4), neuromusculoskeletal and movement relation functions (b7) and functions of the skin and related structures (b8). Emotion functions (b152) were the most frequently linked code, referring to WTs aimed at supporting the monitoring and self-management of emotions (k = 10). Studies were also frequently linked to functions and structures adjoining the eye (b215), which included devices that measured gaze behavior, such as fixations or saccades (k = 8). Involuntary movement functions (b765), were linked to devices which both measured and aimed to reduce stereotypic behavior (k = 6). Seven studies measured attention functions (b140), with a further seven measuring heart functions (b410) including heart rate and pulse. Dispositions and intra-personal functions (b125) and global psychosocial functions (b122) were both linked to four devices and, referred to devices which sought to target or improve social functioning or social-cognitive responding. Chapters 5 and 6 referring to functions of the digestive, metabolic and endocrine systems, and genitourinary and reproductive functions respectively were not represented.

Activity and participation codes referred to the physiological and behavioral functions measured by devices, their functional targets and the context in which the WTs were used. Eight of the nine activity and participation chapters were covered, including learning and applying knowledge (d1), general task demands (d2), communication (d3), mobility (d4), self-care (d5), interpersonal interactions and relationships (d7), major life areas (d8), and community, social and civic life (d9). The most frequently linked code was focusing attention (d160), which included devices measuring and facilitating eye contact or face-to-face behavior in autistic youth (k = 14). Communicating with receiving nonverbal messages was also commonly linked (k = 9), referring to devices measuring and facilitating the facial emotion recognition, while managing one’s own behavior (d250) referred to devices seeking to support self-regulation (k = 5). Engagement in play (d880), school education (d820) and copying (d130) linked to seven, five and three devices respectively referred to the context in which devices were commonly used. Domestic life (d6) was not represented.

Environment codes were linked to the context that the WTs were used in and covered three of the five environment chapters of the ICF including products and technology (e1), supports and relationships (e3), and service systems and policies (e5). WTs were frequently used with the child’s immediate family (e310) (k = 14), with other professionals (e360) including researchers or observers (k = 13) and with health professionals (e355) such as therapists (k = 7). Devices were also used with acquaintances, peers, colleagues, neighbors and community members (e325), and people in positions of authority (e330), such as teachers which were linked to six devices each. Devices were commonly linked to products and technology for education (e130), which included devices that were paired with technologies such as robots for the purposes of therapy, as well as devices that provided coaching (k = 11). Four devices were linked to education and training services (e585) and three to health services, systems and policies (e580) referring to devices used in the context of school or therapy. The natural environment and human-made changes to the environment (e2) and attitudes (e4) were not represented.

WTs to support function are generally concerned with measuring physiological or behavioral activity and using this as an index for function. This process is however rarely a one-to-one mapping exercise, particularly for more psychologically relevant constructs, as was the case for many WTs included in this review as demonstrated by the ICF linking. With these constructs, there is a risk of affirming the consequent. For example, distress will typically result in lower heart rate variability however, if heart rate variability is observed via a WT, it may not have necessarily been preceded by increased distress.

The internal validity of WTs remains largely unknown and researchers seeking to develop WTs to measure and support physiological functioning must be cautious of this fallacy.

Assuming that devices are capable of measuring and indexing the desired functional outcome, the WTs must be generalizable to other contexts. As evidenced by the significant variability in activity, participation, and environmental ICF codes linked to the WTs, it is apparent that many studies sought to employ their WTs in ecologically valid environments and contexts. However, the external validity of nearly all the studies included in this review could be considered low due to their small sample sizes. Autistic individuals exhibit high variation and cannot be well represented by the small samples used in case-studies and case series studies, contributing to difficulty in understanding the true effectiveness and reliability of the WTs described.

The devices must also be able to reliably collect physiological or behavioral data. As ASD is a heterogeneous condition, with its effects on functioning influenced by a number of factors, including the environment (8), it is essential that WTs are readily adaptable to variability in individual functioning. While not within the scope of this review, various physiological measurements may be derived from WTs (for example, heart rate variability, beats per minute, respiratory sinus arrhythmia), with varying levels of accuracy and reliability. The reliability of measures taken from WTs may be influenced by a number of factors, including their positioning on the body, such as in the case of electrodermal activity, which may show an asymmetric pattern of activity, resulting in different readings of arousal depending on the side of the body the measurement was taken from (66). Future research should seek to conduct thorough reliability testing of these devices (65, 67).

A strength of many studies was that the experiences of both the WT users and caregivers were taken into account, with some studies reporting the development of their devices in collaboration with autistic individuals themselves (44, 58). These demand and consumer-driven approaches enable developers and researchers to ensure the needs of autistic youth and their families are met. Future research should continue to seek the perspectives and experiences of autistic individuals, and their families in the design of WTs. However, a cautionary note must be made in relation to a ‘one-size-fits-all’ approach to wearables in ASD. This is particularly relevant given that ASD is inherently a spectrum disorder and the wide acknowledgment that each autistic person is unique, with differing behaviors and needs. It is likely that the customizability of wearables is critical in supporting their acceptability and fostering of positive functional outcomes (68).

Some studies reported that the aesthetics of devices were a concern for participants. Beyond considering the properties of weight and obtrusiveness, researchers largely failed to consider the implications of the size or appearance of the wearables within a child’s everyday environment. This is of particular relevance in ASD were wearing devices that are socially obtrusive may further exacerbate social difficulties (68). Many emerging WTs are sleek and discrete, and aligned with fashion trends (i.e., smartwatches and bracelets), increasing their social desirability and acceptability, enhancing their usability and ability to empower autistic youth. Ideally, in the future wearables for autistic youth will have the capacity to sustainably address the needs of a child from functional, aesthetic, technical and cultural viewpoints (69).

While studies included in this review demonstrated that the WTs were generally tolerated and accepted by participants, few examined the impact of wearing these devices for long periods. Those that did often reported discomfort after 20-30 minutes due to heating or weight of the device (33, 34, 37), suggesting that there are still challenges to their prolonged use. Very few devices also exhibited the ability to store or process information retrieved in the WTs themselves, requiring a personal computer or a central processing unit for the individual or experimenter to access the information. Optimization of all aspects, including the aesthetics of WT is paramount to ensure that WTs are capable of meeting the needs of autistic youth.