On Electronic Travel Aid Design

In order to navigate, we must first know "where" things are, in other words, the spatial positions of objects and other features in the physical world around us. Thus, we can reframe the problem of traveling as initially a problem in spatial perception. Man is well endowed with a refined set of spatial sensing systems, and listed in order of decreasing range of operation, they include binocular vision, binaural hearing, and active touch. To compensate for the loss of binocular vision, a blind traveler desires some sort of electronic travel aid (ETA) that can spatially sense what is out there in the environment out to a reasonable distance, and then display this information in an easily understood format via the remaining senses of hearing and touch. So first, let's consider the basic problem of spatial sensing and how it is normally done.

Spatial Sensing

a) First is the basic problem of "sensing" itself. Generally, in order for me to detect an object, it must be a source of energy, either generated by the object itself or reflected. For example, a sighted individual usually sees objects by reflected light, such as sunlight or the light generated by a lamp. Or I can see something that is generating light, such as this computer monitor that I'm writing on. One exception to the general rule cited above, is that an object can be detected even though it is not a source of energy, as when I detect the waving my hand in front of my monitor because it blocks a source of energy known to be present.

b) Second, we now consider the problem of "spatial" sensing. Our senses, such as the eyes or ears, can infer the direction of an object because we assume that the energy being radiated, travels in straight lines. In the case of the eye, energy arriving from each angular direction, be it azimuth or elevation, is directed by the lens to a two dimensional array of photosensors on the retina, each direction being mapped to a unique point on the array. However, we live in a three dimensional space. This means that three coordinates are required to specify spatial position, not only azimuth and elevation, but also range. Thus, nature provides us with another sensor a few inches away, in other words a second eye, so we can calculate range using binocular vision. Generally speaking, all three quantities need to be known, or your movement will be impaired. In case you don't believe me, try rock climbing while wearing an eye patch. Without going into all the details, some of which are still not yet known, range appears to be calculated via a process akin to triangulation, (since I know the distance between my two eyes, and the two angles specifying direction from each eye to the object).

In the case of electronic travel aids for the blind, spatial sensing is to be done electronically. Past approaches include ultrasonic air sonar, rangefinding via laser triangulation, and most recently GPS (global positioning system) using radio triangulation of timing signals from simultaneously viewed satellites.

Display parameters

Presuming that we can satisfactorily do electronic spatial sensing, the next problem is how to display the spatial information to the blind user. Since hearing and touch are the two remaining entry points for information to the blind traveler, let's consider how these two systems operate in man.

We perceive sound as having the following attributes: 1) loudness (intensity), 2) pitch (frequency, repetition rate), 3) phase (with respect to a reference), 4) timbre (spectrum or equivalently the waveform shape as a function of time), and 5) spatial location (direction and range). Typically, information gets encoded upon sound via by passing it through a physical space of a particular shape. For example, when I talk, my vocal cords open and close generating a train of impulsive-like broadband sounds which passes through the vocal tract, whose shape changes as a function of time depending upon what words I wish to say. Similarly, with musical instruments, the shape of the cavity will determine the pitch and timbre typical of the particular instrument. In spatial hearing, the asymmetrical shape of the external ear or pinna will cause a different transformation to be applied to incoming sound, depending on its direction of arrival and range. In each case, the auditory system can determine these spatial transformations, so that we can understand the speech or music, or perceive a sound's location.

In terms of tactual perception, the skin has some similarity to the ear, and can sense vibrational stimuli (changing pressures) with attributes including: 1) intensity, 2) frequency, 3) phase (when two or more points are stimulated), 4) waveform shape as a function of time and 5) location (and shape) of stimulation on the skin's surface. In addition to vibration, we can perceive other kinds of stimuli including 6) temperature (heat or cold), 7) pain or tickling, 8) electrical stimulation and 9) chemical attributes, such as irritation or taste. Moreover, combining the above sensations with proprioceptive cues yields "active touch," which means we can discover the shape of an object by actively moving the skin's sensory surface over it, and feel the changing sensations as a function of position and orientation.

In the case of ETAs, a wide variety of electronically generated sound and tactual cues have been used to signal the blind user. In the case of hand held narrow beam devices (ultrasound or laser ranging), azimuth and elevation cues are typically conveyed proprioceptively via hand orientation. The coding of range information is the most difficult for the blind person to understand. Most often, ETAs convey range information either by a vibration or sound that increases/decreases in frequency and/or intensity as the detected object gets closer. More recently, some experimental ETAs may inform the blind user the exact range, or even location via synthesized speech.

Head mounted devices use similar cues, except that it is head orientation that conveys azimuth and elevation information. For the devices that have a wider angle of view, an additional cue can be used to signal azimuth, such as the binaural interaural intensity difference. Even TV cameras have been hooked up to large area tactual displays, which convey range information in a somewhat surprising manner: as an object comes closer in range, the area of tactual stimulation becomes larger. I have designed systems that use more naturalistic sounds such as the pinna tranformational cues of spatial hearing and the human echolocation cue of time separation pitch.

For more details on individual devices, and links to other organizations presently doing research in ETAs, see my web pages:
Currently available ETAs
Some ETA Patents

Other interesting links in this field:
ASMONC: Autonomous System for Mobility Orientation, Navigation and Communication
Blind Mobility Research Unit
Seeing Sounds: Echolocation by Blind Humans, by Christine Uy
Sonification of Range for 3-D Space Perception
Multipath Systems, Inc.
Perceptual Alternatives, by Tony Heyes
External links by Peter Meijer
Tips for Inventors of ETAs

Some links to other organizations that work with the blind and visually impaired:
American Council for the Blind
American Foundation for the Blind
American Printing House for the Blind
Helen Keller National Center
National Federation for the Blind
Smith-Kettlewell Eye Research Institute

Foundations of Orientation and Mobility

last updated 1 November 1998
Copyright © 1996 -8 by Duen Hsi Yen, All rights reserved.
E-mail: yen@noogenesis.com