FlashSonar: Understanding and Applying Sonar Imaging to Mobility

FlashSonar: Understanding and Applying Sonar Imaging to Mobility

Future Reflections Winter 2011
(back) (contents) (next)
FlashSonar: Understanding and Applying Sonar Imaging to Mobility
by Daniel Kish, MA, MA, COMS, NOMC
From the Editor: Consciously or unconsciously, most of us who are blind gather a great deal of valuable information from the echoes that bounce off objects in the environment. Daniel Kish has done extensive study on echolocation and has pioneered methods for training blind children to use it more effectively. Daniel is cofounder and president of World Access for the Blind, a California-based organization that focuses on developing innovative approaches to improving the functioning of blind people. He holds master's degrees in psychology and special education, and he is a certified orientation and mobility instructor.
On my first day of first grade, the bell rings and all the kids scamper gleefully away. I amble after them, occasionally clicking my tongue, listening for the wall to my left and avoiding chairs left askew. I hear kids laughing and shouting outside through the open door. I hear the sides of the doorway in front of me, and I center myself as I pass through it to the new playground beyond. After a few steps I pause to consider the strange, chaotic environment stretching out before me. I stand on a crack that runs parallel to the building behind me, where the smooth cement turns to rough pavement. I wish my feet were not covered with shoes.
I have no cane; mobility isn't provided to children my age in 1972. I have been clicking to get around for as long as I can remember. Everyone says I'm really good at it, but I never think about it. It comes as naturally to me as breathing. I click and turn my head from side to side, scanning the expansive space before me, straining to penetrate the heavy curtain of commotion. The world suddenly seems bigger than anything I've ever encountered, and noisier, too--teeming with flocks of darting voices, swarms of bouncing balls, and battalions of scuffling shoes. What is around me? How do I get there? What do I do when I find it? How do I get back?
I find the noise oppressive, like a looming wall that seems almost impenetrable. But curiosity wins out, and I step cautiously forward, clicking quickly and loudly to cut through the cacophony. I follow the clear spaces, passing between clusters of bodies, keeping my distance from bouncing projectiles. From time to time, I click back over my shoulder. As long as I hear the building call back to me through the crowd, I know I can find it again. However, its presence is fading fast. The noise undulates all around me like a thick pall of fog enveloping my head.
The storm of noise goes on forever in all directions, and I will soon lose the building. Should I head back? A ball skitters behind me and shoes pelt after it. The sounds spur me onward. There must be grass and quiet somewhere, open space like there was on the kindergarten playground.
The pavement starts to slope slightly downward. The building is lost to me now, but I realize that if I find the slope and follow it back upward, it will point me in the right direction. The pressing din gives way to a softer hue, and my clicking inquiries find no reply, suggesting that a very big field of grass lies ahead. With relief I speed up, eager to find open quietude. My shod feet find the grass, and the heavy fog releases me.
Stimulated by the promise of adventure, I break into a run, quickly clicking to ensure that nothing stands in my way. I'm free as a bird taking joyful flight. Then, suddenly, something whispers back to me from the open expanse, and I jolt to a stop. "Hi," I venture in a bell-like treble. There is no reply. As I scan, clicking more softly, the something quietly tells me about itself--it is taller than I am and too thin to be a person. When I reach out to touch it, I know already that it is a pole. I'm glad I found it with my ears and not my head. The pole has a small metal cap on top. I click around me, and barely hear something else whispering back. Leaving the pole, I move toward this next thing as it calls to me with a similar voice, telling me that it is also a pole. I detect yet another one, and another--nine poles in a straight line. Later I learn that this is a slalom course. In time I practiced biking by slaloming rows of trees while clicking madly.
A buzzer abruptly slices the air. I am not startled, but I freeze and raise my hands to my ears. When it finally ceases, I lower my hands to hear buildings from far away calling back to the buzzer. I detest the buzzer, but the distant voices echoing back sound like wistful music. I scan around me, clicking, but I can't hear the building over the great distance and bedlam of kids. I clap my hands with a sharp report, and something large calls back through the tangle of piping voices and scurrying shoes. I turn in that direction. The grass gives way to pavement, and as I step quickly up the slope, clicking and clapping, I hear the unmistakably broad, clear voice of a wall drawing nearer.
The crowd noise has organized itself and is not quite so assaultive. I hear kids in lines facing the wall. I don't know why they're lining up or what I'm supposed to do, and I can't tell where my classroom is. The wall sounds completely featureless, offering no information. I ask someone a question, and someone points me in the right direction. I start to walk along the crack parallel to the wall, but kids are standing on it. I move in toward the wall, clicking and walking between it and the fronts of the lines until someone calls my name. I find the right line and, turning away from the building, I click my way along the line until it runs out of bodies, now all quiet as directed by the teacher. I lay my hands on the shoulders of the kid in front of me as I was taught--a boy I would guess by his T-shirt and short hair. As the line moves and we enter the room, I let go, clicking and scanning to avoid kids as they shuffle into their chairs. I click along the wall to my right until I near a corner. Sensing the distance from the wall in front of me, I know I'm near my desk at the end of my row. I reach to my left and find a desk with a Braillewriter on it. I take my seat, wondering how big the new playground is, and if it has a slide. I wriggle with excitement to find out more next recess.
Perceiving the Environment
Through our perceptual system, the brain constructs images to represent everything we experience in our conscious minds. The way we interact with the environment depends upon the quality of these images. When vision is disrupted, the brain naturally works to maintain image quality by optimizing its ability to perceive through other senses. The brain seeks to discover and explore in order to heighten the quality of meaningful information gathered through our experiences. The inability to see with our eyes need not be disabling when the brain learns to "see" with an intact and heightened perceptual imaging system. Indeed, the visual system of the brain is recruited to assist in processing nonvisual stimuli such as echoes and tactile information. Our approach to long cane and FlashSonar training is thus based in perceptual science in order to activate the imaging system quickly and efficiently.
Cane travel and other areas of perceptual training are integral to our approach to orientation and mobility. If I could redo anything about my childhood, it would be to have a long white cane available to me. We have developed approaches to long cane training for children at their first steps and before. However, this article focuses on FlashSonar, as we feel it is the least understood and most poorly implemented element in standard mobility training.
Both sight and hearing interpret patterns of energy reflected from surfaces in the environment. Reflected sound energy is called echo. The use of echoes, or sonar location, can help a person perceive three characteristics of objects in the environment--location, dimension (height and width), and depth of structure (solid vs. sparse, reflective vs. absorbent). This information allows the brain to extract a functional image of the environment for hundreds of yards, depending on the size of the elements and strength of the sonar signal. For example, a parked car, detectable from six or seven yards away, may be perceived as a large object that starts out low at one end, rises in the middle, and drops off again at the other end. The differentiation in the height and slope pitch at either end can identify the front from the back; typically, the front will be lower, with a more gradual slope up to the roof. Distinguishing between types of vehicles is also possible. A pickup truck, for instance, is usually tall, with a hollow sound reflecting from its bed. An SUV is usually tall and blocky overall, with a distinctly blocky geometry at the rear. A tree is imaged according to relatively narrow and solid characteristics at the bottom, broadening in all directions and becoming more sparse toward the top. More specific characteristics, such as size, leafiness, or height of the branches can also be determined. Using this information in synergy with other auditory perceptions as well as touch and the long cane, a scene can be analyzed and imaged, allowing the listener to establish orientation and guide movement within the scene.
Passive and Active Sonar
There are also two types of sonar processing--passive and active. Passive sonar is the most widely used type among humans. It relies on sounds in the environment or sounds casually produced by the listener, such as footsteps or cane taps. The images thus produced are relatively vague and out of focus. Passive sonar may be sufficient for detecting the presence of objects, but not for distinguishing detailed features. It's a little like hearing the murmur of other people's conversations around you. You catch bits and pieces, but the information contained therein may or may not be relevant or discernible.
Active sonar involves the use of a signal that is actively produced by the listener. It allows the perception of specific features as well as objects at greater distances than passive sonar. It's more like engaging in active conversation with elements of the environment. One can ask specific questions of particular elements and receive clearer answers. In fact, scientists who study bats call the process of bat sonar "interrogating the environment." The bat is actively involved in querying features of the environment for specific information through an array of complex sonar calls almost as varied and strategic as a language. Only recently has it been made clear that humans can learn to do likewise.
Because of its relative precision, active sonar is used most widely in nature and in technical applications. The greater accuracy of active sonar lies in the brain's ability to distinguish between the characteristics of the signal it produces from those of the returning echo. The echo is changed by the environment from which the signal bounces. These changes carry information about what the signal encounters. In our work with blind students we use the term FlashSonar because the most effective echo signals resemble a flash of sound, much like the flash of a camera. The brain captures the reflection of the signal, much like a camera's film.
Perhaps the greatest advantage of FlashSonar is that an active signal can be produced very consistently and the brain can tune to this specific signal. Elicited echoes can easily be recognized and small details detected, even in complex or noisy environments. It's like recognizing a familiar face or voice in a crowd. The more familiar the face, the more easily it is recognized. The characteristics of an active signal can be controlled deliberately by the user to fit the requirements of a given situation, and the brain is primed to attend to each echo by virtue of its control over the signal.
Discerning the Signals
Tongue clicks can be used effectively to gather sonar information about the environment. The click should be sharp, similar to the snap of a finger or the pop of chewing gum. It can be very discreet, no louder than the situation dictates. Hand claps or mechanical clickers may be used as a backup, but they require the use of the hands and are not easily controlled. Clickers are generally too loud for indoor use. They should never be sounded near the ears, and never clicked more than once every two or three seconds. Cane taps can be used in a pinch, but the signal is poorly aligned with the ears, and it is inconsistent as surface characteristics change. The use of cane taps in this way may encourage unnecessarily noisy or sloppy cane technique.
We find that sonar signals are rarely noticed by the general public, so they do not constitute a concern against normalcy. They generally result in improved posture, more natural gait and head movement, greater confidence, and more graceful interaction with the environment.
When we teach FlashSonar to students, we start by sensitizing them to echo stimuli. Usually we have them detect and locate easy targets such as large plastic panels or bowls. The idea is to help the student get a sense of how echoes sound. We call this a "hook stimulus" because it hooks the brain's attention to a stimulus that it might otherwise ignore. Once this recognition is established, we gradually move to subtler and more complex stimuli.
We use stimulus clarification to help a student perceive a stimulus that he/she may not sense, such as an open door or a pole. To clarify the stimulus, we may use a large pole or a wide doorway, or use a reverberant room beyond the doorway. Once the student can detect the clarified stimulus, we return to the original stimulus.
Our most frequent approach is stimulus comparison. We exemplify the sounds of environmental characteristics by using A-B comparisons wherever possible. For example, solid vs. sparse may be shown by comparing a fence near a wall. A high wall could be found near a low wall, or a tree near a pole, or a large alcove near a smaller one. We try to locate training environments that are rich with stimulus variation. The characteristics of almost any object or feature can be better understood when compared to something distinctly different.
Stimulus association is the conceptual version of stimulus comparison. Instead of comparing elements in the environment, we are comparing real elements to those in our minds by drawing upon mental references. For example, when facing a hedge, a student might say, "It sounds solid."
I might reply, "As solid as the wall to your house?"
"No, not that solid," she might say.
"As sparse as the fence of your yard?"
"No, more solid than that," she might answer.
Now we have a range of relativity to work with. "Does it remind you of anything near your house, maybe in the side yard?"
"Bushes?" she might query.
"What seems different from those bushes?"
"These are sort of flat like a fence."
If she still can't put words to what she is perceiving, we tell her what the object actually is--a hedge. Ultimately, we have students verify what they hear by touching and exploring.
We also encourage precision interaction with the environment. For instance, we might have a student practice walking through a doorway without touching, with the door closed more and more to narrow the gap, or having a student locate the exact position of a thin pole, and reach out to touch it without fishing for it. We also work on maintaining orientation and connectedness with surfaces in complex spaces. A good example of this is moving diagonally from one corner to another across a very large room, like an auditorium or gym. They learn to hear the corner opening up behind them, while closing in before them, and keeping their line between the two. The world is not made of squares and right angles, but of angles and curves. This exercise helps stimulate the ability to process nonlinear space. It is surprisingly difficult for many students, but surprisingly easy for others. Once this task is mastered, we place obstacles to be negotiated while still maintaining orientation.
Freedom to Explore
Ultimately, we support students to be able to orient themselves and travel confidently through any space, familiar or not. We practice finding and establishing the relative locations of objects and reference points in a complex environment, such as a park or college campus. The students walk through the area, keeping track of their location with respect to things they can hear and echolocate. They are discouraged from staying on paths, but urged to venture across open spaces. We find and map objects and features until the space is learned. Active echolocation makes this process go much faster.
The most important thing is to allow and encourage blind children to explore their environment without constantly supervising their every move or structuring all of their activities. It is important that they often direct their own movements, not relying strictly on direction from others. The occasional hint is nice, but spoon-feeding our kids all the answers is debilitating; it breaks down the perceptual system. We must remember that the brain is like a muscle. It only gets stronger with self-directed exercise. The earlier this happens in a child's life, the more comfortable and friendly will the child's relationship become with the environment.
The mother of a boy we have worked with wrote to us about her son's progress. I quote from her letter:
"My youngest son, Justin, totally blind, is five ... we introduced Justin to a white cane when he was eighteen months old, ... [and] he ... processes the information he gains from it very effectively. Justin is a very active, outgoing fellow who loves socializing and sports of any kind. ... the work [Daniel] has done with Justin has had tremendous results. ... Walls are easy for Justin to hear. He has moved on to identify parked cars, store displays, other solid objects like newspaper boxes, bushes, and more, all with the click of his tongue. ... If I ask Justin to go and find a ... solid object that doesn't make noise, he will click his tongue and ... set off in that direction. As he nears it, he will actually pick up speed and become more confident. ... He can then stop short of it. ... The delight on his face when ... he discovers what he has found is unparalleled. ... The other day my husband asked Justin to tell him when the type of fence changed along the street. ... Clicking his tongue, Justin could tell him when the fence changed from brick to wrought iron. ... we had seen Justin using echolocation on his own as a toddler. ... I'm not sure how much Justin knew what he was doing, or how much further he would have taken it. I know that I have heard a lot of blind adults say that they use echolocation to some degree. ... But in Justin's case, with structured training, his potential in this area is being drawn out and he is learning to use echolocation more effectively than he would have otherwise. ...
"Probably my greatest strength ... is my ability to teach him social skills. I have a very strong interest in this area, and it shows in who Justin is becoming. He is extremely well spoken, ... very outgoing, confident, and well-liked by his friends and classmates. ... a tongue click ... is hardly noticeable. In fact, unless you were listening specifically for it, I don't know that you would notice it. Okay, if you are blind you almost surely would, but I am commenting as a sighted person. It is hardly noticeable at all. ... The tongue click in no way resembles a blindism or mannerism. ...
"Today Justin does not exhibit any mannerisms. ... Here is something positive that [echolocation] does do. It keeps the head up nicely, because when you click to scan your environment you lift your head up instead of hanging it down. ... We, like any parents, want the best for our son. We want him to be as independent and free as he can be. To give him that, we want him to have access to all the options so that he knows what is possible and can make his own choices. ... I strive to give my child access to all of the resources I can to help him become who he wants to be. This is one such resource. ... Echolocation training is most definitely helping to accomplish that goal.
--Tricia"
Try It Yourself
For sighted parents, close your eyes and have someone hold a bowl or open box in front of your face. Speak. Listen to the hollow sound of your voice. Now have the box removed and put back. Hear how your voice sounds open or closed in. Try the same thing with a larger box or a pot. Hear how your voice sounds different between smaller and larger objects--perhaps deeper for the large pot.
Try the same thing with a pillow or cushion, and notice that your voice sounds soft instead of hard. Try going to the corner of a room and hear how your voice sounds hollow when you're facing the corner. How does the sound change when you face the center of the room?
For further information and video demonstrations of our FlashSonar program, please visit <www.worldaccessfortheblind.org>.
(back) (contents) (next)