Technical Aspects of the Development of a Web-based, Interactive Nemeth Code Tutorial

By Gaylen Kapperman, Jodi Sticken, and Larry Skutchan

Gaylen Kapperman is a professor emeritus of Northern Illinois University where he has been the director of the Visual Disabilities program for over three decades. Kapperman is blind and is a member of the National Federation of the Blind. He holds a doctorate from the University of Northern Colorado. In addition, he had been awarded a Fulbright Fellowship for study at the University of Heidelberg, Germany.

Jodi Sticken is Director of the Orientation and Mobility Program in the Visual Disabilities Program at Northern Illinois University.

Larry Skutchan is Director of Technology Product Research at the American Printing House for the Blind, Inc.

Abstract

The authors describe the steps involved in the development of the Nemeth Code Tutorial, a web-based tutorial for blind persons to learn to read and write the code of braille mathematics.

Keywords

Technology, mathematics, tutorial, Nemeth Code, braille mathematics

 

In partnership with the staff of the American Printing House for the Blind (APH), the staff of Research and Development Institute developed the Nemeth Code Tutorial, an innovative, highly effective tool in the form of a web-based, interactive tutorial that blind individuals may use to become proficient in reading and writing braille math symbols. The tutorial became available in February 2015. In previous research, we have demonstrated that the use of such a tutorial is extremely effective (Kapperman, Sticken, & Smith, 2011, 2012). The major problem at this time is that many individuals who are blind cannot access the tutorial that is still available, as it functions exclusively on one specific and remarkably expensive stand-alone device.

The tutorial which we have developed is a tool which is available on the APH website at no cost. It is accessible to the vast majority of individuals who are blind using the full range of mainstream technology in such a manner that it is compatible with new and developing technology. As technology continues to evolve, this robust tutorial will continue to be a functional, valuable tool for blind people for many years to come.

The Challenge

Employment forecasts indicate that an increasing number of career specialties will require that individuals have expertise in technical fields. All technical fields require facility in mathematics, which is a difficult area of study for individuals who cannot see—particularly if an individual is not familiar with the Nemeth Code, the braille code of mathematics. Unfortunately, at this time and for the foreseeable future, many individuals who read braille will not have the opportunity to learn the Nemeth Code through the conventional teacher-student method due to the shortage of qualified teachers who have this competency and limited access to the independent study method caused by the current existence of few Nemeth Code tutors (DeMario & Lian, 2000; DeMario, Lang, & Lian, 1998; Kapperman, 1994; Kapperman & Sticken, 2003; Rosenblum & Amato, 2004; Wittenstein, 1993).

The web-based tutorial is readily available to the braille reader who cannot access mathematics because of a lack of knowledge of the braille math code. When individuals who are blind are proficient in reading and writing braille mathematics, they have the potential to acquire the quantitative skills necessary for career training in the technical fields. Thus, access to the Nemeth Code Tutorial will eventually result in greater employment opportunities for individuals who are blind.

The Nemeth Code Tutorial

The tutorial consists of 55 different lessons. Each lesson contains four sections. The first is an explanatory section in which a specific set of Nemeth Code symbols are described; the rules governing the use of those symbols are described in detail along with examples. There are three interactive sections in each lesson following the explanatory section: reading, writing, and proofreading. In the reading section, the learner is given a Nemeth Code math expression and is asked to read it. The user can invoke the proper key combination to have the screen reader speak the expression in order to compare their response to the correct response. This can be done as many times as the learner wishes. The second interactive section deals with writing Nemeth Code math expressions. In this section, the learner is given a math expression in written (literary) form, and is asked to use six-key entry or a braille display to input the expression in Nemeth Code. The response can then be graded. The software will provide feedback as either correct or incorrect (depending on the accuracy of the response). The learner may repeat the exercise as many times as desired to have each subsequent response graded. The third section is dedicated to proofreading. Here the learner is given a Nemeth Code expression in words along with the expression in braille. The braille expression contains one or more errors. The learner is instructed to find the errors and correct them using mainstream editing commands; this response can then be graded. The exercise can be repeated as many times as required to determine the correct response.

The content of the lessons range from the very simple beginning symbols, such as the numeric indicator and punctuation indicator, to the complex symbols found in higher level mathematics such as algebra, trigonometry, geometry, calculus, logic, and set theory.

We have previously carried out three different, well-designed studies which all indicate that this approach is extraordinarily effective (Kapperman & Sticken, 2003; Kapperman, et al., 2011, 2012). All three studies resulted in the same conclusion. When blind learners use the interactive tutorial in learning to read and write the braille code of mathematics, they make statistically significant growth scores compared to a matched group of learners who do not use the tutorial. We know it works!

Technical Development of the Tutorial

The tutorial has been developed with assistive technology that is widely used by people with visual disabilities. The Nemeth Code Tutorial works with nearly any modern browser/screen reader combination on nearly any platform. Interestingly, browser support of MathML is a deciding factor for sighted users. Since Internet Explorer and Chrome, at the time of this writing, do not fully support the Math ML markup language, visual representation of the more complex math expressions is rendered better in the Firefox and Safari browsers. As Internet Explorer (and its successor) and Chrome continue to add support for interpreting math expressions encoded in MathML, the tutorial software engineers anticipate that the performance of these browsers will improve requiring no changes to the tutorial itself. In the meantime, programmers added support for Math JAX, a JavaScript display engine for all browsers. Sighted users can also still employ the browsers that do not fully support MathML by using the literary equivalents associated with this type of math content. As a simple example, if the browser does not support MathML, a fraction is expressed as “three over four” instead of showing the numerator, denominator, and fraction line normally associated with such an expression.

For users who are blind, the tutorial works well with any screen reader that includes support for a refreshable braille display. While it is possible for a learner who is blind to use the tutorial without a refreshable braille display, it is not recommended because the Nemeth Code is better interpreted through braille rather than speech. On the Windows platform, these screen readers include NVDA, JAWS, and Window-Eyes. These screen readers are listed in order of best support (at the time of this writing) for both MathML and support of the Unicode character set, two features that improve their utility with the tutorial. VoiceOver is the only available screen reader for iOS on the iPhone and iPad and for OSX on the Mac; however, the tutorial works with this screen reader as well. One minor drawback to using VoiceOver is that there are not separate input and output braille translation settings. Therefore, the user has to turn off translation to perform Nemeth Code input for the exercises. iOS 8.3 adds this additional capability. For Android tablets, the TalkBack screen reader also works. The Chrome operating system is also supported through the use of the ChromeVox screen reader.

Any refreshable braille display is compatible with the tutorial as long as the display is supported by the user’s screen reader. There is no attempt to restrict the use of this educational material to any specific product of any vender. Because of this approach, the tutorial will be available to the largest number of persons who read braille and anyone using the most up-to-date assistive technology commonly available to persons who are braille readers.

The following is a description of the procedures for developing the Web-based Nemeth Code Tutorial. There are two approaches that are technically sound for the development of such a product. One approach involves using existing screen reading technology; the other requires writing components and adapting other existing components. The advantage of using screen reading technology is that the end user presumably already knows the commands to control the access technology. Speech and braille drivers are already developed and installed on the user's system, while screen readers are effective in rendering web-based output in a meaningful way.

Some of these advantages, unfortunately, are also disadvantages. The handling of web controls, for instance, is different for each screen reader. Moreover, the behavior for a particular control may not be optimal for this tutorial. If a screen reader were used, it could be turned off for web interactions thus allowing the tutorial software to use what it needs from the screen reader while controlling the behavior itself. However, this requires that instructions be provided which explain how certain features of the specific screen reading software may be disabled. As a result, this option would cause an additional burden for the user. In many cases, the user may not be sufficiently technically sophisticated to easily employ the special instructions.

The custom software approach includes many characteristics that make it the more effective choice. This approach enables the tutorial to operate in the same manner no matter which technology the learner employs. Also, this approach eliminates potential user confusion when special instructions must be followed to avoid screen reader limitations. Unfortunately, it also introduces the difficult problem of creating speech and braille display drivers that function within a browser or requires the user to install those drivers. The development group chose a method that is a combination of both. The tutorial uses standard screen reader commands and provides feedback via Accessible Rich Internet Applications (ARIA) tagging.

To create the tutorial, programmers employed modern web standards such as JavaScript, HTML5, and ARIA. To show the braille dot configurations to sighted users, they used Unicode braille characters. This allows the braille to be displayed visually as well as being rendered correctly by screen readers without requiring the user to download an additional braille font. To permit sighted users to input their answers in braille for the exercise portions of the tutorial, developers created a six-key input system. The tutorial is hosted on the Drupal content management system, and the JavaScript is generated with PHP. Since MathML and ARIA support is relatively new to screen readers, a frequent schedule of rapid iterations was employed. This allowed the engineers to identify problems early and provide corrections in a timely fashion.

The tutorial uses MathML if the user's browser agent or screen reader supports it. Employing MathML provides the sighted user with quality renditions of complex expressions. For those screen readers that support MathML (NVDA, JAWS, Window-Eyes, and VoiceOver), the tutorial allows the screen reader to render the expressions in Nemeth Code. For those browsers that do not support MathML, the tutorial shows the expressions written in plain English. For screen readers that do not support MathML, the tutorial stores and shows its own answers pre-encoded in Nemeth. If the user checks the “Use MathML” checkbox and the screen reader supports MathML, the user sees both the Nemeth Code generated by the screen reader and that which was already encoded. It is not feasible to develop a Nemeth Code back-translation system at this time, since all expressions are hand translated into Nemeth Code and marked up appropriately so that they display properly in both print and braille.

Functions were written for the equation editing portion of the tutorial in order to enable the user to move the cursor to any location in the equation at any time while displaying the braille symbols properly. Also, if the user must scroll up or down line by line, proper alignment will be maintained. In this way, despite the absence of multiple lines on the braille display, the user can move from line to line without affecting the horizontal position of the cursor. It was necessary to implement this function because most electronic refreshable braille displays contain only one line.

Additionally, to facilitate navigation to various portions of the equations, routines were also developed along with the assignment of keystroke bindings. Translation tables and character code indexes from the screen reader are used so that the number of characters per symbol assists the user in navigating through the various portions of the equation. If, for example, the user presses the right arrow key to move to the next character of the expression, the braille cursor will move the appropriate number of characters to bypass the intervening symbols no matter how many intervening characters exist.

If the user wishes to interact with individual symbols that comprise a Nemeth Code expression, the cursor routing keys on the braille display may be used to accomplish this task. Software engineers also included an editing mode in which the regular keyboard arrow keys can be used to move between individual braille cells for some portions of the lessons where editing the Nemeth Code is the goal rather than navigating from entity to entity in an expression. Most electronic refreshable braille displays contain a button above or below each braille cell to enable the user to immediately move the cursor to an individual cell. This can be done even if the Nemeth Code representation of a symbol comprises several cells. Most screen readers include support for reading these cursor routing keys. This functionality can be combined with the translator and knowledge about each of the Nemeth symbols to place the cursor in the appropriate location when using one of the cursor routing keys to edit a symbol or expression.

While the user may employ the braille input keyboard found on many refreshable braille displays, he may instead simulate a braille keyboard by using six keys on the regular QWERTY keyboard. Software engineers developed functions to capture and gather multiple key inputs and a test to ensure the keyboard was able to type up to six keys at once. Bluetooth keyboard support was also added. This functionality operates by displaying to the user a "Use Six-Key Input" button on the tutorial's home screen. Once the user checks this box, the tutorial instructs the user to press the six keys (s d f j k l) simultaneously to test the capability of his keyboard to accept the pressing of multiple keys at once. If the keyboard is able to accept the pressing of multiple keys, the tutorial software checks the box and saves this information in a cookie, allowing the user to perform this test only once per machine.

Functionality with Dated and Current Assistive Technology

The Nemeth tutorial website is designed to work with both past and current technology. If the learner employs a user agent that does not support JavaScript, the ARIA prompts will not automatically notify him in real time about errors. However, the user may activate a "Show Answer" link to check his work. Similarly, if the blind user's screen reader does not support ARIA, the link may also be activated to obtain feedback about the correct answer. As screen readers and browsers are updated, these additional features will begin working.

For those browsers that support MathML, the user sees a properly rendered version of the expression. If the browser does not support MathML, the tutorial displays a text equivalent. Most browsers already support the Unicode character set which allows the sighted user to see the braille without requiring a separately downloaded braille font. At the same time, most screen readers also show the proper dot configurations for Unicode braille characters. One current drawback to this approach occurs in the VoiceOver screen reader. Currently, if the user turns off braille translation in order to properly input Nemeth Code symbols, the braille Unicode representations are not shown properly on the braille display. This requires that VoiceOver users be instructed to turn the eight-dot mode on or off, depending on the portion of the tutorial on which they are currently working.

Future Plans

Before the adoption of Unified English Braille (UEB,) it was not feasible to expect screen readers and translation software to provide accurate representations of many of the math expressions shown in the Nemeth tutorial. The current implementation avoids this problem. For example, the expression “3 + 3” is displayed as "three plus three." However, currently the developers of screen readers are implementing the capability of their products to use UEB to display some expressions in that code. In the former code, for example, a simple expression like 3 + 3 had to be translated into braille that contained the literary number 3, the word "plus”, and the literary symbol for the number three again.

While this approach works well for learners who are blind and use speech synthesis as their primary mode for learning, it presents a distinct disadvantage for learners who are sighted. Teachers of visually impaired students and potential braille transcribers who are learning the Nemeth Code would find the presentation more valuable if the actual braille math symbols were used rather than their literary counterparts. While many screen readers and browsers of the past were not equipped to handle these types of representations, recent screen reader technology, braille standards, and browser capabilities have advanced enough since the release of the original tutorial to permit such a presentation. One minor drawback to this approach is the additional burden of configuring the user’s screen reader to announce punctuation and math symbols if the synthesized speech output is utilized. During the initial transition to UEB, it may also be somewhat difficult for screen reader users to interpret some of the more complex expressions translated to UEB. The very existence of such problems, however, emphasizes the extreme need for a system like the Nemeth Code which is capable of expressing the most complex mathematical concepts in braille.

The interactive tutorial is installed on the APH site and ready to be used. This revolutionary tool can be accessed by anyone at any time, free of charge, at http://tech.aph.org/nemeth.

This project was funded by a grant from the U.S. Department of Education (grant no. H133G110122). No opinions or judgments expressed in this document are those of the U.S. Department of Education. All are those of the authors.

References

DeMario, N., & Lian, G. (2000). Teachers’ perceptions of need for and competency in transcribing braille materials in the Nemeth Code. Journal of Visual Impairment & Blindness, 94(1) 7-14. 

DeMario, N., Lang, S., & Lian, G. (1998) Teachers’ self-assessed competence and attitude toward literary braille and Nemeth Code. Journal of Visual Impairment & Blindness, 92(5), 354-357. 

Kapperman, G. (1994). [Survey of knowledge of the Nemeth Code by special educators and teachers]. Unpublished raw data. 

Kapperman, G., & Sticken, J. (2003). A case for increased training in the Nemeth Code of braille mathematics for teachers of students who are visually impaired. Journal of Visual Impairment & Blindness, 97(2), 110-112. 

Kapperman, G., Sticken, J., & Smith, T. (2011). The effectiveness of the Nemeth Code tutorial for the BrailleNote. Insight: Research and Practice in Visual Impairment and Blindness, 4(1), 22-28.

Kapperman, G., Sticken, J., & Smith, T. (2012). The effectiveness of the Nemeth Code tutorial for the BrailleNote: A follow-up study. Insight: Research and Practice in Visual Impairment and Blindness, 5(4), 168-174.

Rosenblum, L. P., & Amato, S. (2004). Preparation in and use of the Nemeth braille code for mathematics by teachers of students with visual impairments. Journal of Visual Impairment & Blindness, 98(8), 484-495.

Wittenstein, S. H. (1993). Braille literacy: Pre-service training and teacher attitudes: Report of a national study. (Unpublished doctoral dissertation). Teachers College, Columbia University, New York.


The Journal of Blindness Innovation and Research is copyright (c) 2016 to the National Federation of the Blind.