Future Reflections Special Issue: Technology
If You Can Imagine It, You Can Draw It
by Richard Baldwin
From the Editor: Richard Baldwin is a professor of computer information technology at Austin Community College in Austin, Texas.
Four years ago a blind student named Amanda Lacy enrolled in one of my computer programming classes at Austin Community College. She was bright, she worked hard, and she did very well in my course. Later I learned with surprise that she was struggling in a physics class. I knew that her difficulties had nothing to do with her innate ability. It turned out that the problem was her lack of access to course materials. Amanda had an electronic version of the physics textbook, but it contained hundreds of equations, pictures, and diagrams that she could not access with her assistive technology--a screen reader and a Braille display.
I felt strongly that blind students should not be excluded from the study of physics due to inaccessible materials. I offered to tutor Amanda, and we worked together for about four hours a week. Eventually I authored an online physics tutorial called "Accessible Physics Concepts for Blind Students."
Diagrams and Doodles
Through my work with Amanda, I learned that blind students needed an effective way to draw and submit graphical homework assignments such as vector diagrams. Although several outstanding free drawing programs were available on the Internet, they were designed for sighted users. Most of them required the user to manipulate a mouse. For Amanda and other blind students, I wrote an accessible drawing program named SVGDraw01. The theme for this program is "If you can imagine it, you can draw it."
Amanda was thrilled when I sent her a preliminary version of the program. "Finally, I can doodle!" she exclaimed. Before she had the program, her physics professor allowed her to skip assignments that required sketches. Working with the new software, she could turn in her homework with the rest of her classmates.
Though the program is still under development, it already allows blind students in science, technology, engineering, and mathematics (STEM) courses to create drawings that mirror many of the figures and diagrams typically found in textbooks. The drawings can be printed for sharing with sighted teachers and classmates. Also they can be embossed for the blind student's own use and for sharing with other blind or visually impaired people.
The user interface for SVGDraw01 is straightforward and accessible with screen readers and refreshable Braille displays. It consists of menus, buttons, checkboxes, and fill-in-the-blanks text fields. For example, Figure 1 shows a form that would be filled in for the purpose of adding a circle to the drawing. This form is exposed by selecting Circle on the draw menu shown at the top of Figure 1.
Titles, instructions, and field contents are announced by typical screen readers, thus eliminating confusion on the part of the user as to what to do next. Pressing F1 at any point brings up the accessible on-screen instruction manual shown in Figure 2. This document also responds well to typical screen reader commands.
Blind and visually impaired students using this program can create both printed and tactile graphics, using the same thought processes they would use to construct a drawing on a graph board with pushpins, rubber bands, a protractor, and a measuring stick. One person might use the program to create and send a drawing file to a friend with the message, "Take a look at the cool floor plan of my new apartment." Another person might use the program to create and send a drawing file to a college professor with the message, "This is a free body diagram showing the magnitude and direction of forces f21 and f23, caused by the interactions among charges q1, q2, and q3."
Whether created by blind or sighted people, drawings usually consist of various shapes with color added for visual enhancement. With SVGDraw01, blind and visually impaired people can compose drawings using all or parts of the following shapes: lines, rectangles, circles, ellipses, and vectors. The program also permits users to add free-form shapes (polylines, polygons, and paths), along with text, to a drawing. Once a shape is created, it can be embellished through rotation, translation, scaling, and clipping. Existing shapes can be deleted from the drawing or reviewed without modification. If need be, attributes of a shape can be modified after it has been added to the drawing. More advanced features are available, including the ability to control the width of lines, as well as the color or gray scale and the opacity of the individual shapes.
All of these features are found in typical drawing programs designed for sighted users. The main difference is that SVGDraw01 does not require the use of a mouse for these features to be accessed. All of the program's features (with the exception of the experimental AudioTac display) can be accessed using the keyboard alone.
Figure 3 shows a doodle that was produced using the program. The doodle illustrates some of the things that can be drawn. When printed on an ordinary color printer to be viewed by a sighted person, the doodle would look as it appears here. If this doodle were embossed using a high-resolution, high-quality Braille printer such as the Tiger Embosser from ViewPlus Technologies, the colors and shades of gray could be made apparent in tactile form through the use of variable dot height. If the doodle were produced on a less capable embosser without variable dot height, the user would need to forego the shades of gray and convert the red, orange, or magenta colors to solid black or leave them white.
The program provides an instant visual display for the benefit of sighted teachers. This feature is also useful for blind and visually impaired students who may want to explore their drawing, using soundscapes such as those produced by the program known as the vOICe. An experimental screen display known as AudioTac is also provided. AudioTac makes it possible for blind and visually impaired people to explore a drawing tactilely with an auxiliary touchpad in conjunction with audio signals. The jury is still out on the usefulness of this feature.
Scalable Vector Graphics
The primary output from the program is a graphics file in a format known as Scalable Vector Graphics (SVG). While many graphics file formats are in common use, SVG is the one common form that makes it possible to enlarge a drawing contained in a file without degrading the quality of the drawing. The user can zoom in on and emboss large-scale versions of complex portions of drawings. Drawing files that were previously created and saved can be imported back into the program for further enhancement or to be used in combination with new or other imported drawings. The SVG file format is currently compatible with Firefox 8, IE 9, Google Chrome 15, and ViewPlus IVEO. The output files are directly compatible with the Tiger line of graphic embossers from ViewPlus Technologies.
Amanda Lacy and I are developing an auxiliary program for free distribution. It will translate the SVG file format into a format called SIG. The SIG format is compatible with embossers that use the QuickTac software from Duxbury Systems, Inc. With the auxiliary program, named JpgToSig-A-01, the user will be able to emboss output files on several brands of embossers, including older models. The program accepts any of several bitmap image files as input and writes the enhanced version of the image into an output SIG file. While it is possible for a blind student to use the program, it is primarily targeted for use by teachers and others who assist blind students.
SVGDraw01 is available for free download and should run on any Windows operating system, Version XP or later. A download link for the program is available at <http://www.dickbaldwin.com>. The program is self-contained, and no special software or complex installation procedure is required.
Embossing Graphics from Textbooks
Following my work on an accessible drawing program, I decided to tackle the inaccessible pictures and diagrams in electronic textbooks. One way blind students can understand diagrams and pictures is by embossing them, using any of several available embossing techniques. Basically, an embossed image is a document containing raised lines or raised dots that reproduce the image's salient features. Originally, I hoped to make it possible for blind students to emboss their own textbook images. For several reasons, however, this is still a dream.
The most common format for electronic textbooks is the Adobe PDF format. With existing software it is impossible for a blind student to extract most of the images in a PDF file intact. Numerous programs claim to extract the images from PDF files, but in most cases each image ends up in several different files that must be reassembled for embossing.
Since bitmap files use some 16 million colors and color combinations, there are about 16 million reasons why the embossed version of a full-color bitmap image often fails to produce satisfactory tactile results. To begin with, the embossing process often discards the information content from more than 16 million colors, ending up with an image that represents black and white or, at best, black and white with two or three shades of gray in between. To add further complications, unless the original image is very small, the spatial sampling will probably be reduced by a factor of five to ten in the embossed image. The bottom line is that it is very difficult to emboss full-color bitmap images and end up with high-quality tactile images.
Different embossing methods produce different physical outputs. Many of the older Braille printers have a graphics mode that allows pictures to be displayed by raising a subset of individual Braille dots to a standard height. The dot separation on those printers ranges from ten dots per inch to seventeen dots per inch. In contrast, the typical image seen on a computer monitor contains ninety-six dots per inch. When such an image is re-sampled down to a level that is consistent with the number of dots in an embossed image, much of the detail simply disappears.
Newer Braille printers have dot resolutions of up to twenty-five dots per inch, which is still very low compared to onscreen images. To simulate gray scale imaging, some of these new embossers can raise dots to variable heights. An experienced blind user can probably recognize dot heights representing black, white, and perhaps three gray levels in-between.
Some embossing techniques may convert the 16 million colors in a bitmap image to black and white through the application of a single intensity threshold. Others convert the 16 million colors to black, white, and several shades of gray through the application of several intensity thresholds. In either case, many colored pixels that are clearly distinguishable in the original graphic are indistinguishable in a four- or five-level gray scale version of the image. Detail that depends on the recognition of different colors simply disappears, and many of the salient features of the image are lost in the embossing process.
I have developed a mathematical image-processing algorithm which, in many instances, preserves much more detail than the typical intensity-based gray scale approach. This algorithm converts the original image either to black and white or to black and white plus three levels of gray--based not on absolute colors, but rather on changes in color. The images processed using this algorithm tend to have black outlines that define the salient features of the original image. In many cases, this method produces more meaningful embossed images than the typical approach based on the direct conversion of color intensity to gray scale.
To solve the spatial sampling issue, my programs make it possible to subdivide an enhanced image into panels. These panels can be individually embossed and then assembled into a poster-sized tactile image. While this is not an ideal solution, it is the best I have to offer at this time. An embossing system with an improved dot resolution is greatly needed.
The algorithm is packaged in a free computer program named ShapeExtractor02. It is designed for use with any embossing method that can accept JPEG image files as input. This program accepts any of several different bitmap image files as input and writes the enhanced version of the image into an output JPEG file. Both of these programs can be downloaded freely along with the program named SVGDraw01. Note that all three of these programs require the Windows operating system.
These programs can be used with bitmap images from any source. However, in the world of education for blind students, the images that need to be embossed are often contained in electronic PDF versions of course textbooks. The following procedure can be used to emboss images from a PDF textbook.
1. Open the PDF file in the free version of Adobe Acrobat and locate the image of interest.
2. Use the zoom capability of Acrobat to make the image as large as possible while still fitting on the screen.
3. Hold down the shift key and press the Print Screen key. This saves the current screen image on the clipboard.
4. Open any of many available image editing programs such as LView Pro.
5. Paste the clipboard into the image editor.
6. Crop the image out of the surrounding material, retaining only the portion necessary to contain the image.
7. Save the cropped image.
8. Open either JpgToSig-A-01 or ShapeExtractor02 and follow the usage instructions to convert the image to the desired black and white or black, white, and gray format.
9. Save the enhanced image in an output file and emboss it, using the embossing method of choice.