Paul Mitten(19995 bytes)
Paul Mitten

Nonvisual Access to Electronic Print Displays

by Paul Mitten

From the Editor: Paul Mitten is Vice President of Compusult Limited, of Canada.

Good afternoon and thank you for inviting me to this conference and for the opportunity to make a presentation regarding Compusult Limited's work in assistive technology, especially as it relates to nonvisual access to electronic print displays. I would also like to touch upon nonvisual access to electronic displays in general, especially with respect to workplace adaptation.

I would like to start with a little background overview, which, of course, must begin in the seventeenth century. Our company, Compusult Limited, is located in Newfoundland, and there is an interesting historical connection between Newfoundland and the state of Maryland, as well as the city of Baltimore, which I would like to relate to you.

In 1620 Sir George Calvert, the first Lord Baltimore, purchased a parcel of land called Ferryland in Newfoundland from Sir William Vaughan. The records do not state whether Mr. Vaughan was heard to laugh upon conclusion of his latest real estate deal.

In any event, in the following year--1621--Calvert's merry band of colonists set off from England for Ferryland. Calvert himself resided at Ferryland for less than two years, from the summer of 1627 until the winter of 1628-29. The diaries of Lady Baltimore include a comment essentially stating that she "could not spend another winter in this Godforsaken place." This excerpt, by the way, is proudly displayed in the Ferryland Museum.

Therefore, by 1629 George Calvert was determined to obtain lands in a friendlier climate. His petition for a large colonial grant located north of the Potomac River was agreed to by King Charles I, but Calvert died almost two months to the day before the charter for Maryland was officially granted, possibly from a cold acquired in Newfoundland. Cecil or Cecilius Calvert, the Second Lord Baltimore and the first son and heir of George Calvert, inherited the title and the 10 to 12 million acres of American land in what became Maryland from his father and served as the designer and first Lord Proprietor of that colony from 1632 to 1675. The first settlers landed on Saint Clement Island (Clement possibly being chosen in reference to the milder climate). Maryland became the fourth English colony and the second plantation colony. This colony, with the Second Lord Baltimore as proprietor, was the first enduring proprietorship in the New World.

If I might engage in some creative hindcasting, as opposed to the creative forecasting undertaken by some of the other presenters during these past two days, I would posit that, without the enduring influence of Newfoundland on Lady Baltimore and hence the First Lord Baltimore, the history of Maryland and this city would be radically different, leading to the distinct possibility that the NFB might not be located in Baltimore, nor would we be here today.

By the way, Ferryland is actually a pretty and friendly little place that holds the Ferryland-Maryland days each summer to celebrate the historic connection. There is also an interesting long-term archeological dig underway to document and recreate Lord Baltimore's Colony of Avalon in Ferryland.

But I digress. Hopefully, Newfoundland will continue to have an enduring influence, especially in the area of Assistive Technology. Compusult has been involved in assistive technology development since 1993, focusing primarily on workplace adaptation to meet the needs of blind and low-vision employees. Compusult's introduction into this world was prompted by the Assistive Devices Industry Office of Industry Canada, which is directed by Mary Frances Laughton. The ADIO previously coordinated a program that provided seed funding for the development of assistive technology intended for commercial use. This program helped build the AT industry in Canada and brought several new products and technologies to the AT marketplace.

The DigiCite Technology

In 1994 Compusult proposed to the ADIO development of a device that would speech-enable LED and LCD displays. We also proposed that this device would be a handheld, battery-powered, portable unit ready for market in less than twelve months. We later learned that our plans rapidly outpaced reality. Despite this concern, we successfully developed the base technology for reading LED displays using a fixed, dedicated system called DigiCite.

In 1997 the National Industries for the Blind (NIB) contacted us regarding our DigiCite technology, which they required to create and maintain jobs for blind and low-vision switchboard operators in VA medical centers and Air Force bases. We conducted feasibility studies at two VA's in 1998 with the assistance of the NIB and their associate agencies. We then began installation of the first systems at these facilities in late 1998 and have continued with new installations since that time. To date DigiCite systems have been used to read LED and VFD displays of various physical sizes and colors in lengths of four, eight, sixteen, and forty characters.

The DigiCite systems take a different approach to speech-enabling electronic equipment. In the past the NIB had sought solutions designed specifically for a given manufacturer's product or product line. However, few manufacturers or third parties offered speech-enabling solutions for much of the equipment concerned. Despite the ADA and other legislation and initiatives, this has not yet changed to a great degree. Many of the new systems, as well as legacy systems, still offer no straightforward means of providing nonvisual access to electronic displays and indicators.

Our approach incorporates an image-acquisition device, i.e., a digital CCD camera mounted on the electronic equipment which is interfaced to a small footprint computer system. Images of the electronic display are captured continuously. The image is analyzed to identify the presence of characters. If the displayed characters are deemed to have changed, a corresponding message is output using digitized speech. Character identification is achieved using custom Optical Character Recognition (OCR) techniques. A DigiCite system must be trained to work with the characters seen on a given display and must usually be trained across multiple units of the same type to ensure the highest level of accuracy.

All processing is carried out on a Corel NetWinder computer system, which is also manufactured in Canada. It uses the Linux operating system on a StrongARM processor, thus providing a very fast, cost-effective, reliable, and stable operating environment for our DigiCite software, which must run twenty-four hours per day, seven days per week at each location. The NetWinder also offers integrated video capture and audio output in its compact package, ideally suited to the DigiCite system requirements.

The DigiCite approach offers the following advantages:

*It has generic application and will work with practically any equipment with an electronic LED or VFD display.

*There are no liability concerns, since all DigiCite equipment is externally mounted and no access to the internals of the equipment is required. When a burly technical sergeant stomps up to you at an Air Force base and asks what you are going to do to his telecomm equipment (which is rather important to base operations and also handles the occasional call from the White House), it is reassuring to say "nothing at all; all of our equipment is external to the consoles and won't impact anything." With that the sergeant tends to leave us alone and let us complete the installation work.

*It accommodates all users; the image acquisition device is mounted to allow easy repositioning for non-blind users whenever required.

*It accommodates equipment replacement and transfer. For example, a switchboard console is replaced in its entirety when considered defective or problematic; the DigiCite arrangement allows for immediate transfer to another unit with relative ease. This transfer has been performed by switchboard staff.

*It allows for external technical support and updates. Remote dial-in access to the DigiCite computer systems allows Compusult to perform troubleshooting, software and configuration updates, etc. all via remote dial-up sessions.

The DigiCite approach currently has the following drawbacks:

*A system is relatively labor-intensive to deliver. All equipment has to be installed and configured by highly-trained personnel with experience and expertise in optics, electronics, digital camera technology, and the related computer systems and software. It is not end-user installable.

*External lighting has an adverse effect. When using a digital camera as the image-acquisition device, there are sometimes undesirable effects on the imagery imposed by the working environment, ambient lighting conditions (and variations thereof), lens and camera performance and occasional variation, variability between LED or VFD displays even on the same models of a manufacturer's equipment. However, we have often overcome these problems using a camera hood to block out external lighting effects.

*There is a short delay (typically one to two seconds) between when an LED display changes and when DigiCite begins speaking. For most situations this is acceptable. However, in an operational switchboard environment all operators are under pressure to respond to incoming calls and callbacks as quickly as possible. Hence, even a brief delay may cause some concern.

*The system will occasionally make mistakes in character recognition but will normally correct itself in subsequent image capture/analysis. This, however, degrades response time and can make for annoying repetition. We have incorporated various techniques in the software that can improve recognition accuracy and reliability, but these can slightly degrade performance.

We have been able to overcome these problems at all sites, although in some cases this has necessitated modifications to the software, mounting arrangements, cameras and lens systems, etc., to achieve consistent operations. Each new site presents a different set of conditions which have ultimately led to improvements in the DigiCite system and have reduced the implementation time and costs, while providing greater flexibility and performance.

Other Compusult Systems

While DigiCite provides nonvisual access to electronic print displays, this is not the only factor that must be considered in the typical working environment for a blind employee. Most electronic equipment, including telephone consoles, has electronic indicators (LED, VFD, and LCD) that provide on/off status information. Compusult's Vindicator system provides the means of speech-enabling these indicators and can work as a completely stand-alone device or with DigiCite to provide a complete solution for a given type of equipment.

In many VA medical centers the switchboard operators are also responsible for monitoring the status of alarms throughout the facility. These alarm panels are all installed in the switchboard rooms, and, although they provide audible output, they do not identify the exact nature of a given alarm. Since many of these alarms signal potential life or death situations, e.g., cardiac arrest, the operator must be able to react immediately to them and follow the correct procedures for each situation.

Compusult's Responder is a machine-vision-based system that provides a continuous watch on alarm panels and other electronic equipment. Whenever an event such as an alarm occurs, Responder acknowledges this and generates a corresponding synthetic- or digitized-speech message to notify the operators. We have been informed that a blind operator can, in fact, respond more quickly than a sighted operator when using Responder, since the sighted operator would normally have to identify the alarm panel and the specific alarm on that panel by sight before taking action. The blind operator can remain in position and wait for the verbal message.

Future Directions

Compusult continues to develop and enhance the DigiCite technology. The primary aims are to incorporate the ability to read LCD and other displays, improve generic applicability through improved image processing and OCR technology, reduce size and cost, and ultimately achieve our original 1996 goal of a handheld device. In this regard we have also been involved in collaborative proposals to expedite development and broaden the reach of the system. We are part of a proposal to the European Commission, in cooperation with organizations from Germany, France, and the U.K., to develop a portable, handheld device that reads electronic displays as well as printed text, including labels on three-dimensional packaging such as retail goods. We expect to hear a decision on this proposal possibly in November of 1999. This would be a three-year development project, with a ready-to-manufacture product upon its conclusion.

We recently completed a grant funding proposal with the NIB to make the current technology more generic in its applicability to various types of displays including LCD and to reduce the overall implementation costs, especially the labor-intensive installation and configuration process. The ultimate goal is to achieve portability for use with various types of equipment at a work site, and ideally, when we deliver a DigiCite system to clients, they will be able to install it themselves.

The NIB also continues to support us in working with a variety of agencies and companies that want to use DigiCite to make legacy and even new telecommunications equipment accessible to blind workers. Each new contract helps to advance the technology.

I must also note that the NFB, which has a demonstration version of DigiCite in its technology center, has passed along information on our systems to other prospective clients.

And, to conclude, coming full circle to Lady Baltimore, I don't think she gave Newfoundland enough of a chance. My wife took a mere twelve years to get used to the weather and appears to enjoy it now. However, she does find an excuse to leave for two to three weeks each February, I have noticed.