Developing an Open Source Sonar Navigation Device

Comparatively few companies pursue assistive technology because the market is small and highly regulated. This makes the devices offered limited in function and expensive, as well as out of reach for many people who need them.

Recently the ability for individuals to create sophisticated custom electronic devices has expanded dramatically. In the last several years the cost of complex electronic components have dropped, become easier to use, and well documented making it possible for people with a hobbyist level of skill to build unique and useful projects. These factors combine to make it straightforward for people without special training to connect sensors, motors, and other electromechanical devices together with sophisticated logic and programming to create complex devices.

With the increasing availability of these technologies has risen the concept of Open Source Hardware, which makes the full design for a device available for anyone to make, improve and modify it to their own use (OSHW Committee, 2011).

The combination of these two concepts, allowing the average person to make advanced devices and the open sharing of complete device designs has positive implications for assistive technology. Starting from free instructions, a hobbyist would be able build a sophisticated assistive device that can be customised to serve an individual’s specific needs, while still costing less than more general commercial offerings.

This project explores the process first by developing a hand mounted sonar navigation device, then offering the complete plans and instructions to the community and finally observing the results of those who chose to build, modify, and use the device.

Through working on a virtual navigation project for people with vision impairment we found that ultrasonic sensors provided excellent navigation cues. After investigating the current state of ultrasonic navigation aids we decided to create our own.

We looked at many of the ultrasonic devices on the market including hand-held devices and ultrasonic canes. Most devices were expensive, costing more than $US1000. A majority of the devices investigated also relied on audio feedback, despite research showing that audio feedback interferes with other audio cues (Calder, 2009) and many of those with vision impairment also having hearing impairment. One of our testers also believed that audio cues were socially stigmatising, unnecessarily drawing attention to the user.

From examination of the design and function it was clear that we could make a device with off-the-shelf components that would not only be at least as effective as those we investigated, but would also be more affordable while still being buildable by an electronics hobbyist.

Design went through a number of iterations. The haptic feedback mode was chosen in order to avoid the aforementioned difficulties with audio feedback. For the initial test we created a headband with ultrasonic range finders paired with vibration motors around the circumference. The vibration motors, similar to that which causes mobile phones to vibrate, increased in frequency the closer it sensed an obstacle in its sensor’s direction. This version allowed navigation through unknown buildings and proved the concept viable, but it was not a practical success. Applying vibration motors to the head proved disturbing when worn for any length of time, and the head mounting was unable to reliably detect objects below waist level, leading to regular impacts with furniture and other low obstacles. It was also unsightly and awkward to wear.

We chose to move the device from the head to the back of the hand. We have experience developing hand-mounted electronics and understood the difficulties of making the device light, secure, and ‘ruggedised’ to allow for hand movement and impacts. Moving the sensors to the back of the hand allowed the user to point it in any direction they were curious about without impeding hand function. The necessarily smaller device reduced the number of sensors from four to two, but vastly increased the field of coverage and the versatility. The rangefinders were each mounted at 20 degrees on either side of the main axis of the hand. When holding the hand horizontally they detected objects to the right and left of the wearer. When holding the hand vertically they detect objects near and far. The back of the hand also allowed it to function similarly to ultrasonic cane devices while still being able to be used separately. Testers who were blind almost immediately began using the device where canes would be impractical, for example, when finding glassware on the table.

We found research indicating that even small vibrations could, over time, cause sensitivity and even nerve damage (Lundborg et al., 1990). One tester also found the constant vibration distracting. Because of these considerations we decided to use pressure rather than vibration to register distance. Where the original version tied vibration frequency to distance, the revised version would use press rubber pads firmly on the wrist when something was near and gently when it detected far objects. Placing a sensor and rubber pad pair on the right and left side of the wrist allowed a correlation of location and pressure, while keeping the pads separate enough to keep the sensations separate. We researched a number of approaches for this physical feedback and found that common microservos used in remote control cars and airplanes were the best choice. They gave the quickest feedback, and used relatively little power, and are durable. The opportunity cost of going from vibration motors to servos was that the device was going to be larger and more complex.

During the component testing phase we also evaluated other kinds of distance sensors, for example, lasers and infrared reflection. We decided to continue using ultrasonic sensors because they worked well in all environments, were easily available, and while making up most of the cost of the device were still comparatively affordable. However, they are not as precise as other, more expensive solutions, such as lasers. The rangefinders create pulses of 40 kHz sound which is well above human hearing but within the hearing range of many dogs including guide dogs. Our informal testing found no measurable response from dogs exposed to the working sensors. Subsequent research confirmed this (Moxon, Allison, & England, 2010).

The part of the device that comfortably mounted the electronics on the back of the hand (the “gauntlet” as we call it) was developed to meet a number of use considerations. It was placed behind the knuckles and forward of the wrist to keep as much freedom in hand movement as possible. We used a minimum amount of material on the fingers and palm in order to keep the sense of touch as free as possible. The gauntlet used Velcro closures to make it adjustable to many different sizes of hand, and to work as well on the right or left hand. Finally we chose Neoprene as our base material for the gauntlet. Although it can be difficult to work with, it is incredibly durable, looks nice, and provides cushioning both to the wearer and the device allowing the device to be handled roughly in all weather.

While testing took place throughout the development phase, most of the early testing was undertaken by blindfolded developers. Two adult test volunteers who were blind were found to help with the fine tuning. One subject was blind from birth while the other lost his sight as a teenager. Both volunteers had experience with canes, guide dogs, and human assistants. One had tried a sonic cane and found the audio feedback not worth the trouble. Both users were given less than a minute of instruction on use before first using the device and were comfortable with it after a minute’s orientation. Both wearers were able to get useful feedback almost immediately and gained proficiency with practice.

Feedback from the volunteers was mostly positive. Most of the changes involved programming of the microcontroller to interpret the sensor data in more intuitive ways. The sensors could occasionally give erratic and momentarily incorrect readings. A number of techniques were applied to smooth the data while still providing timely feedback to the wearer. Changes were also made to prevent the occasional twitching of the pressure servos, which all wearers found unhelpful (Figure 1).

Figure 1.

The original version of the gauntlet did not support the electronics well enough causing them to flop and catch on things. The thumb loop was replaced with a loop on the ring finger which provided better anchoring of the forward portion reducing slip. The loop was also reported to be more comfortable.

Other changes included swapping the types of Velcro from hook to loop to prevent snags and skin scrapes, and using a lighter weight Neoprene to keep the hand cooler. It was also decided to make the electronics easily removable from the Neoprene so that the gauntlet could be easily laundered or changed. Again Velcro was chosen for its strength and flexibility.

The resultant device could be built for around $US80 and could be built in a weekend by anyone with soldering iron skills (Figure 2).

Figure 2.

After completing the initial design we published on our web site full plans for the device including a demonstration video, development notes, full plans, schematics, illustration, parts list, programming code, and a pattern for the gauntlet. These were released under a license that allowed free copying, redistribution, modification, and also allowed individuals to make their own.

Response was strong and immediate. Within 24 hours more than 10,000 people had visited the page, and it received coverage by national and international media. We also received a large volume of email from people who were blind as well as sighted individuals who were interested in helping family, friends, and community members with vision impairment. The nature and volume of the feedback clearly showed the demand for affordable and accessible assistive technology.

Not all feedback was positive, however. The most common complaint was that the individual did not know someone who had the skills to construct the device. Another common complaint was that it was difficult or impossible for a person with vision impairment to make one for himself.

In the months following its release a number of people built Tacit devices based on our original plans. Several people freely contributed improvements in the design and function. Others have reported making custom versions for their individual needs, such as an illuminated pointer to help an assistant identify objects that they sense, and adding a switch for shorter range finding to help location objects on a table. Other builders simplified the design to have only a single sensor and servo, making the design more compact.

Because we are only publishing plans for the device, this project has not gone through governmental approval common for most assistive devices. This makes it incumbent on the developer and builder to make sure the device is safe and effective. We have done our best to design something that maximises safety and effectiveness, and we believe that even in the worst case the chance for personal danger is virtually nonexistent. However, with the Open Source Hardware model we have no control over those who build the device, or the changes they make in the design. A number of the custom versions made changes that disregarded our research. A common change was to simply attach the device to a glove rather than build a gauntlet. While not dangerous the change somewhat compromises the sense of touch on the worn hand. Another, common, but more serious change was to use vibration motors even though research showed the possibility of numbness and nerve damage from doing so.

From the perspective of the individual builders these changes made sense since they simplify the construction. However, we were surprised that makers would so easily disregard the research and benefits of the design.

While it is impossible to know exactly how many people have chosen to build or use the device we collected feedback from the 23 builders who contacted us to share their product.

In general we consider the project a success. We produced plans for a device that when properly built is a small fraction of the price of similar devices. In sharing the plans we found that many people were introduced to a new class of accessibility devices that they were unable to have before. Finally by incorporating improvements from the community a higher quality device was the result. Unique, custom devices were created to serve the needs of the individual.

The main drawback of the project is that the devices can be built in ways that might endanger the user. While we went to lengths to explain the safety of certain features there is no way to prevent someone from ignoring them when they build their own device.

We are now working on a second version of the project that will possibly be produced as a kit. The kit will be much easier to build, allowing the devices to get ‘on the hands of’ more people who need them, discouraging those who would take possibly troublesome short cuts, but allowing those who want to customise the design to do so.

While the results are not exclusively positive, we think Open Hardware assistive devices, if carefully designed, provide much more help than harm and we will continue to develop the concept further.