Finding direction by sound

Matt Cota

They arrive in different colors, contours,
and shapes revealing where we are and
warning what’s up ahead. While those
with sight can trace where they’re
coming and going, it’s not so easy
when you are blind.

    That is the focus of Dan Jacobson’s research. He has developed a software program that uses sounds
to help blind people understand where they are. “It’s not so much that generally speaking blind people are
immobile, they just don’t know what’s around the next corner or what the street sign says or how to read
the map of Kosovo in the newspaper.”

    But with the use of Jacobson’s computer sound map software, any map that
can be seen can also be heard. “It would enable you and I or anyone that wasn’t an expert to download a map
of say San Luis Obispo and say this is downtown, this is the hills to the west, and this is Cal Poly.”


    With the help of a virtual reality mouse, a blind person will also be able to feel the contours of the
map, all while a computer voice gives directions.”


    There are maps printed in braille for the blind, but the problem is there is a limited amount of information
that can be printed on a braille map. Another problem is that while every blind person would like to use a map,
only about ten percent of the blind can read braille.


    The sound map is still in its development stage.

To top of page

                              Press Release: March 1999

                         Dan Jacobson
                         Sound maps for the visually impaired

                         Maps, a highly visual medium for conveying information, will soon be available as
                         sound maps for the visually-impaired, with computerized audio and tactile features
                         that will move users through landscape features, streets and buildings via their ears
                         and fingers. The digital maps can be accessed via the Internet on a personal
                         computer.

                         Blind users who have tested the sound map prototypes obtained better and faster
                         information than a control group using only tactile maps, reports Dan Jacobson, a
                         geographer and visiting fellow at the University of California-Santa Barbara. He will
                         present his research on Wednesday, March 24 at the 95th annual meeting of the
                         Association of American Geographers in Honolulu, Hawaii.

                         The soundmap system uses a conventional computer with web browser software
                         and a "touch window." By touching adjacent areas of the touch pad, users explore
                         the size and shape of adjacent map features. "Earcons," tone-based symbols
                         combining pitch and rhythm, provide instructions for navigating around the map.
                         Environmental sounds (such as traffic noise to indicate roads) and spoken location
                         names also aid navigation.

                         All of the ten testers of the prototype, five visually impaired and five blind, were
                         able to successfully navigate the map after fifteen minutes of instruction, even those
                         who had never before used a computer.

                         Jacobson is working with a haptic mouse, a device that delivers force feedback
                         through the hand holding the mouse. The haptic mouse offers a three-dimensional
                         impression of a virtual map surface through the applications of different haptic
                         effects. For example, a "virtual wall" would offer slight resistance. Requiring extra
                         force from the user in order to pass through, this innovation allows perception of
                         the shape and layout of the map features. An opposite effect, called a gravity well,
                         pulls a user into a predetermined object.

                         Jacobson's work is supported by a grant from UC-Santa Barbara and a Mary E.
                         Switzer Fellowship from the U.S. Department of Education's Office of Special
                         Education and Rehabilitation, National Institute for Disability and Rehabilitation
                         Research.

To top of page




University of California Press Release 'New computer sound maps will help blind', April 1999


Press Release April 1999
 

                     NEW COMPUTER SOUND MAPS WILL HELP BLIND

                     A new technology will help blind computer users with special sound maps that
                     allow them to move through landscapes, diagrams and graphs by using their ears
                     and fingers, using a personal computer.

                     The new auditory maps, being developed by researchers at the University of
                     California, Santa Barbara, rely on a glass "touch window" with a calibrated electric
                     current that can be activated by the touch of a finger or stylus. By touching the glass
                     which lies flat like a mousepad, users can trace across streets, buildings, lakes, and
                     other landscape features, activating sounds, like traffic, as well as spoken words,
                     such as street names.

                     The system also employs "earcons," a play on the word "icons," which are
                     tone-based symbols combining pitch and rhythm. Earcons provide additional
                     instructions for navigating around the map.

                     Dan Jacobson, a post graduate researcher in the Department of Geography who
                     developed the new system, recently presented the research at the national meeting
                     of the Association of American Geographers. Jacobson explained that there is a
                     huge need for these sound-based maps, since only 10 to 12 percent of blind
                     people read Braille.

                     He said that sound maps can improve on the daily living problems facing blind
                     people by leading to higher quality of life through enhanced orientation, mobility and
                     independence.

                     He reported that ten testers of the prototype, five visually impaired and five blind,
                     were able to successfully navigate the map after only fifteen minutes of instruction.
                     This included individuals who had never before used a computer. The blind users
                     who tested the sound map prototypes also obtained better and faster information
                     than a control group using only tactile maps.

                     The testers gave the sound maps high marks. One 37-year-old visually impaired
                     female said, "It was very easy to use, you could stop, take your time, or go back to
                     get information. It helps you to build up a mental picture of the area. It would be
                     good for mobility, if you wanted to get arounda  new area, and build up a picture of
                     how things are, in relation to each other."

                     Jacobson mentioned that the increasingly visual nature of the World Wide Web has
                     frustrated many people with limited vision in their efforts to access that information.

                     The system uses a conventional computer with web browser software, and is
                     perfectly suited for adaptation to the World Wide Web. The hyper-lined
                     multimedia sound maps are layered and linked so that the user can zoom in on
                     certain features, getting to know all the details about a certain portion of landscape,
                     or many landscapes.

                     "It's almost as limitless as the web behind it. Allowing multiple users to access any
                     one map from anywhere With a suitable internet connection" said Jacobson who
                     plans to design a prototype for lease to schools. "There are many applications in
                     addition to use by the blind, for example, sound maps may be helpful to dyslexics,
                     young children, and foreign language speakers." A set of cognitive and perceptual
                     experiments Are planned to provide guidelines for conveying graphics through
                     Sound and touch.

                     Along with the touch window, Jacobson is also pioneering the use of a haptic
                     mouse -- haptic refers to sense of touch. The haptic mouse is shaped like a
                     conventional mouse but is connected to a rod and a box. It's movement is based on
                     a fixed frame of reference, and gives feedback to the user via various effects such
                     as the feeling of a washboard surface to indicate certain types of terrain, or a feeling
                     of resistance that defines a "virtual" border or a wall.

                     In order to disseminate the technique as widely as possible The research team is
                     working on software tools that would enable sighted users to easily convert
                     conventional computer graphics to auditory maps.

                     Jacobson is joined in this research by an interdisciplinary team of researchers
                     including, Reginald G. Golledge, professor of geography; Mary Hegarty, associate
                     professor of psychology; JoAnne Kuchera-Morin, professor of music and
                     associate dean of Computing and Technology, College of Letters and Science; Dr.
                     Stephen Pope from the music department; And graduate and undergraduate
                     students from the Computer Science department.

                     The work is supported by a special grant called Research Across Disciplines or
                     RAD, from the Office of Research at UCSB; and by the Mary E. Switzer
                     Fellowship from the U.S. Department of Education's Office of Special Education
                     and Rehabilitation, National Institute for Disability and Rehabilitation Research.

To top of page