Computer Haptics: A New Way of Increasing Access and Understanding of Math and Science for Students Who are Blind and Visually Impaired

By Marjorie Anne Darrah

Dr. Darrah is currently an Associate Professor of Mathematics at West Virginia University and the co-founder of eTouchSciences, LLC, a company that creates virtual reality educational software for students with visual impairments.

Abstract

Often detailed visual information is used to present math and science content. This can take on many forms in the classroom from textbook pictures to computer simulations. These visual presentation methods are not readily accessible to visually impaired students and this can lead to a lack of understanding and concept development. The students may not understand what they are missing and the teacher may not know how to easily convey that information. An innovative technology, computer haptics, provides a way to easily offer additional information through the sense of touch to supplement information being provided through auditory and visual means. Using a computer and a peripheral device called a haptic force-feedback controller students can virtually explore three-dimensional shapes and receive tactile and kinesthetic sensations (i.e. shape, weight, viscosity, texture, etc.) This article outlines preliminary research in testing multi-sensory learning materials that incorporate computer haptics, auditory cues, and high-contrast visuals.

Keywords

Haptics, Visually Impaired, Middle School Science, Middle School Math, Virtual Reality

 

Acknowledgement

This project has been funded at least in part with Federal funds from the U.S. Department of Education under contract number ED-IES-11-C-0028. The content of this publication does not necessarily reflect the views or policies of the U.S. Department of Education nor does mention of trade names, commercial products, or organizations imply endorsement by the U.S. Government.

Introduction

The National Federation of the Blind (2011) website reports that there are 93,600 school age children in the United States who are blind. The formal learning setting for students with visual impairments has changed over the last 50 years. Ferrell (2007) notes that in the 1950s, over 88% percent of children who were blind were educated in separate or residential schools and that by 1999 this number had dropped to around 73%. Major changes in public policy and educational placement were enacted over the years so that as of 2007, the majority of students (87.2%) with visual impairments are now served in regular schools in the U.S. (US Department of Education, 2005).

Students with low vision or who are blind are often disadvantaged when presented with information in the regular classroom. This is specifically true for curricular areas such as math and science, which often rely on visuals to convey key aspects of the content. There are a number of methods that provide accessibility, such as 2D tactile graphics, 3D physical models, video description, etc. However, most often these items are not readily available for every child due to cost and many other factors. The use of a low-cost computer-based technology can be an alternative means to provide access to information for people with visual impairments. There are thousands of computer programs for the education of sighted children, but very few computer-based educational applications for students with vision problems. The need for innovative assistive devices and techniques is highlighted in a study by Smith, Geruschat, and Huebner (2004). Computer haptics, coupled with audio and high contrast graphics, can allow students who are blind or have low vision to take advantage of a multimodal learning experience. 

Although computer haptic technology has been on the scene since the early 1970s, with the first patent issued to Thomas D. Shannon (1973), this technology has not been widely utilized in the K-12 classroom. The adoption of such technology into mainstream acceptance and use may still be several years away. In order to facilitate the acceptance and use of haptic-based products by teachers and students who may benefit from them, the research and development described in this paper is based on the user-oriented development (UOID) model (Burkman, 1987), which focuses on perceived attributes as central to the development of instructional products that users would find valuable and be willing to adopt. The components of Burkman’s model include: (1) identify the potential users; (2) measure relevant potential adopter perceptions; (3) design and develop a user-friendly product; (4) inform the potential adopter of the product’s user friendliness; and (5) provide post-adoption support.

The techniques of computer haptics are explained in the next section. There is also a description of several of the haptic-based computer applications that have been developed, with a full list in the appendix, and the particular haptic device being utilized throughout the study. In addition, the purpose and methods of the research and development are presented, along with the findings to show usability and effectiveness of these products with students who are blind or visually impaired.

Computer Haptics

Computer haptics is an emerging technology that provides force-feedback and tactile sensations to users as they interact with a virtual object. Haptic hardware provides sensory feedback that simulates physical properties and forces. Just as the monitor enables sighted users to see computer-generated images and audio speakers allow users to hear sounds, the haptic device makes it possible for blind or visually impaired users to feel force feedback and textures while they manipulate virtual two and three-dimensional objects. The haptic device allows the user to interact with a virtual object, such as a planet surface feature or a cell membrane, using the sense of touch. Other physical properties can also be simulated, such as textures, magnetism, viscosity, vibration, or elasticity.

Researchers and practitioners have utilized haptic technology in a learning environment for many years. In their article, “Haptics in Education: Exploring an Untapped Sensory Modality”, Minogue and Jones (2006) give a comprehensive examination of the existing research in haptics and its potential impact on education. This paper, based on Minogue’s doctoral research, summarizes 43 empirically-based peer-reviewed journal articles, three empirically-based books, 11 theoretically-based peer reviewed articles, and 31 theoretically-based books. Minogue and Jones (2006) point out that there have been over 200 articles written on the application of haptic technology in the medical field since 2000. These authors outline many studies pointing to the effectiveness of using haptics to increase learning outcomes. The article also points out that although “haptics is an exciting and innovative way to enhance the learning environment” there are still impediments to the adoption and widespread use of haptics in education. They classify these impediments as perceptual, technological, and methodological and they call for more research in each of these areas. In the perceptual area, they state that further exploration into the nature and persistence of visual-haptic information interactions is warranted. In the technological area, they suggest that the advancement of the haptic devices and providing them at lower cost will be the keys necessary for the technology to be suitable to enter the classroom. Finally, in the methodological realm, Minogue and Jones (2006) call for more research studies that are able to control for confounding variables to assess the cognitive impact of haptic technology.     

Sjostrom (2001) formulated preliminary recommendations for incorporating haptic technology in computer interfaces specifically for users who are blind or visually impaired. The guidelines have been demonstrated to improve performance related to navigation, finding objects, understanding objects, and physical interactions. Some of Sjostrom’s recommendations include: (1) providing well-defined and easy-to-find reference points, (2) avoiding changing reference systems unnecessarily, (3) compensating for one-touch-point haptics devices by enlarging interaction points or using magnetic objects, and (4) providing paths to trace in order to find the objects in the scene.

In another study by Brewster (2002), users who are blind or visually impaired were able to accomplish graphical data visualization tasks when the data is presented through haptic and audio channels. The use of non-speech sound and force feedback from the PHANTOM Desktop haptics device significantly improved the interaction of blind students with graphs and tables. Additionally, the use of the Logitech Wingman Force Feedback Mouse and web audio features allowed users who are blind and visually impaired to successfully extract web-based graphical information (Yu, Reid, & Brewster, 2002). And more recently McGookin and Brewster (2007) have developed and evaluated an application that allows users who are visually impaired to interactively construct bar graphs using a similar haptic device called the PHANTOM Omni haptic device. Although initial evaluation with testers showed that the graph builder was superior to other accepted graph construction techniques used by this audience (corkboard techniques), the authors point out that classroom testing is still needed.

In one study where science-related haptic software was used with students with visual impairments, the researcher found that adding forces (simulated by haptic feedback) to the visual display enhanced users’ understanding of the binding energy of a drug molecule (Brooks, 1990). More recently, Jones, Andre, Superfine, and Taylor (2003) investigated the effects of integrating haptics into a school biology lesson on viruses. The students were able to interact with the viruses on a microscope slide through the sense of touch. This proved to be a huge step toward accessibility for science content that was not previously accessible to students who are blind or have low vision.  

Haptics have also allowed students to interact with “invisible phenomena” in the physical sciences. Clarke and Jorde (2004) studied the impact of integrating computer generated tactile feedback into an inquiry project for 120 eighth graders. In this study, tactile feedback was associated with middle school students’ improved understanding of thermal equilibrium. Also, a NASA Langley Project has developed and piloted software to help students learn about simple machines (Williams, Chen, & Seaton 2003). This project used a low cost force feedback joystick and included five interactive activities to reinforce concepts presented in the standard simple machines curriculum. Some of the labs included force feedback and others did not. The greatest educational gains were noted for the labs that included the computer generated force feedback. 

For our research, software-based materials, including haptic effects, auditory cues, and high contrast visual objects, were developed to deliver middle school math and science content. The haptic-based self-contained software applications, which are aligned with state and national middle school math and science content standards, teach students one or two main concepts per application. The target audience for these materials is middle school students who are blind or visually impaired.

Examples of the Haptic-based Applications

The goal of this research and development effort was to design and implement twenty different computer-based applications (apps) that cover selected math and science topics. A complete list of the apps is in the Appendix, Table 1. Each of the applications includes a Teacher Lesson Guide that outlines content standards that the app will address, guiding questions for the app, an overview of the lesson, and how the lesson could be used in the classroom. The apps that were tested during this initial research are described below:

  • Three Dimensional Shapes: Students are presented with three different textures to feel (i.e., smooth, gritty, and bumpy). They are also presented with common items that are three different shapes (i.e., spheres, cubes, and cylinders). Each of the shapes is displayed with the different textures for a total of nine different scenes with objects to feel and identify. For example, there is a smooth ball, a gritty cement block, or a bumpy log to feel. By the time the student has completed the Three Dimensional Shapes Lesson, they will be acquainted with how to use the haptic device and how to interact with virtual objects in a scene.
  • Exploring the Atom: Students learn the three different parts of an atom (i.e., electron, proton, and neutron) with various forces and textures for users to feel so they can distinguish between the three parts. Students are then presented with the atoms of three different elements to interact with. They learn that the protons and neutrons are in the nucleus, while the electrons are in rings around the outside of the atom. They can feel and count the number of the parts in the hydrogen, carbon, and nitrogen atoms, while receiving audio cues and hearing interesting facts about these atoms.
  • Gravity on the Planets: Students learn about mass and weight and how they are different. They feel how the weight of the same bowling ball changes on the different terrestrial planets and the moon, while its mass stays the same. Students can use the haptics device to feel the size and shape of the bowling ball and can lift it to feel its weight on the terrestrial planets and the moon.
  • Surface Area of a Cube: Students feel a cube and interact with its faces. They explore an unfolded cube to develop the formulas for surface area. They compare how the surface area changes when the side length doubles. They then practice using the formula they have discovered. 

The Novint Falcon® Haptic Controller

The applications above were developed for the Novint Falcon. The Falcon is a rugged, low-cost haptic controller that provides high-fidelity three-dimensional force feedback. The controller moves right, left, forwards, and backwards (like a mouse), but it also moves up and down. When a user holds the Falcon’s detachable grip and moves a cursor to interact with a virtual object, environment, or character, motors in the device turn on and are updated approximately 1000 times a second, letting them feel haptic effects such as texture, shape, weight, dimension, and dynamics. The Falcon provides control and interaction with a virtual environment in a realistic way. 

The Falcon was chosen for this research and development effort because of its low cost (around $250) and the fact that it is a robust gaming device that has been on the market for about seven years. This device is about one tenth the price of the next cheapest device. The researcher utilized other devices in previous projects (Darrah, 2012) and found that the more expensive devices which provided additional degrees of freedom (namely roll, pitch, and yaw) were not necessary for the applications in this area. Using the Falcon device, students can sufficiently interact with the 3-D environments in the apps described above.

The Falcon is attached to a PC computer by using a standard USB drive. The specifications for the computer are minimal (1.0 GHz processor, Windows [XP, Vista, 7, 8], 128Mb graphics card, DirectX, 1.5 GB hard drive, 512 MB RAM, and a USB 2.0 connection).  Device drivers can be downloaded from the Novint website.  

Purpose of Study

The developers of the haptic-based math and science materials recognize that the acceptance of the use of computer haptic technology to deliver science concepts, or to be viewed as an assistive device to deliver this content, may meet with resistance from teachers and administrators. However, given the need for access to certain types of data by students with visual impairments, the use of haptic-based software may prove to be a viable and successful alternative. Computer haptics provides additional stimulation for students with visual impairments and engages another learning modality.

The research presented in this paper was based on several assumptions. The first assumption is that there is a need by students with visual impairments to have access to math and science information and data, which is currently presented visually through a textbook, computer programs, video, or computer simulation. Additionally, the researcher believes that haptic technology can provide information through the sense of touch that is comparable to scientific visual information and that all tactile learners will benefit from information being presented through the sense of touch. The researcher also believes that a haptic force-feedback controller can be utilized as an assistive device to provide access to things that cannot be seen.

Current and past projects have demonstrated the benefits of using haptic technology in educational settings. This study addressed three main research questions about the new materials that have been developed to present math and science topics:

Question 1: To what extent does the software function in the manner it was designed to function?

Question 2: Do the haptic-based apps adequately promote learning of science content from national science and mathematics standards to students who are blind or visually impaired?

Question 3: Can the haptic-based apps be effectively deployed by classroom teachers?

Methods

Prior to data collection, approval for this study was obtained through Ethical and Independent Review Services, a free-standing company that provides institutional review board (IRB) services. The process for creating and testing the haptic-based student materials is summarized in the following paragraphs.

Four initial computer applications were developed to be tested: Three Dimensional Shapes, Exploring the Atom, Gravity on the Planets, and Surface Area of a Cube. The function of these apps is described in the section above titled Examples of the Haptic-based Applications. To develop these apps, a lesson was first designed by a teacher outlining what objectives should be covered in the app and how these objectives fit with the national math and science standards. Second, a storyboard was developed outlining each scene of the app and describing the visual, haptic, and audio content that the student would be receiving in the app. Once the storyboard was complete and agreed on by both teacher and developer, an initial version of the application was programmed using 3D GameStudio, incorporating the haptic dynamic link libraries provided by Novint Technologies.

After creating an initial version of a haptic-based computer application that all team members believed to be clear, accurate, and engaging, seven expert reviewers did a quantitative and qualitative review of each app. These experts were adults who were blind, teachers of the visually impaired, science and math teachers, and computer developers. The experts completed an online survey consisting of Likert scale statements with accompanying open-ended questions for comments that focused on functionality and content. These reviews revealed interface weaknesses, resolved interface questions, and revealed pedagogically undesirable aspects, unexpected desirable features, and subtle programming bugs. Subsequent revisions and improvements were made based on these reviews. 

Next, a group of four visually impaired students tested each app. The developers observed the students as they completed the apps. In addition, the students were interviewed about their experience with using the apps. The comments from the interviews were compiled and reviewed by the design team, who then made any necessary revisions. Suggested revisions were made to the apps before they were tested in the classroom. 

To test student learning, the apps were tested first in a classroom setting at the West Virginia School for the Blind, and then in an informal learning setting called Space Camp for Interested Visually Impaired Students (SCI-VIS). In both settings, a within-group pre-test/post-test study was conducted to assess student learning resulting from the use of the apps. A benefit of this design is the inclusion of a pre-test to determine baseline scores. For the classroom setting, two lessons were selected for testing and a short feasibility survey was completed by the teacher. In the informal setting, three lessons were tested with 20 students. 

Participants and Data Collection

Usability testing was conducted with two populations – students and adult “experts.” The term “expert” was used to mean a person who is blind or visually impaired, works with blind or visually impaired individuals, is a subject matter teacher, or has experience with haptic software. The haptic software was reviewed for content and usability by seven adult expert reviewers, including professionals from three different schools for the blind, one middle school, one university, and one software development company. Their backgrounds included 508 compliance specialists, teachers of the visually impaired, science teachers, computer science professors, software developers, and individuals who are blind or visually impaired. 

The first four computer applications were also reviewed for usability by four students with varying levels of visual impairments. One of the reviewers was ten years old and in the fourth grade, while the remaining three were seventeen-years-old and in the 12th grade. Three of the reviewers were male and one was a female. Two have severe visual impairment or severe low vision (i.e., 20/200 to 20/400), one has moderate visual impairment or moderate low vision (i.e., 20/70 to 20/160), and one has profound visual impairment or profound low vision (i.e., 20/500 to 20/1,000).

Each of the adult expert reviewers was sent a haptic device and was emailed a link to download the haptic software to their personal computer. Some adult reviewers downloaded the software onto their home computers, while others downloaded it onto their school computers. The settings for the student expert reviews varied. One reviewer reviewed the apps in his home, one reviewed the apps in a library conference room, and two reviewed the apps at their schools. For the student expert reviewers, the developers supplied the device and laptop computer. 

Feasibility testing was completed with one science teacher using two of the apps with seven students in three classes at a residential, academic school for the blind. In addition, these seven students tested the first two apps for usability. From this pool of students at the school there were only four students whose parents returned the permission form for their students’ pre/post-test data to be used in the study. The teacher indicated that the other parents never returned any forms sent home from the school. Of the four students who were given permission by parents for their pre/post-test data to be used in the study, two were boys and two were girls. One student is totally blind, one has near total visual impairment or near total blindness (i.e., more than 20/1,000), one has severe visual impairment or severe low vision (i.e., 20/200 to 20/400), and one has mild or near vision loss (i.e., 20/30 to 20/60). Three of the students were 12 years of age with two in the seventh grade and one in the sixth grade. One student was 13 years of age in the eighth grade.

The informal learning setting student testing took place in a room at SCI-VIS in Huntsville, Alabama. Students were asked to arrive a day early at SCI-VIS to assist with the testing of the computer applications. Twenty students participated in the testing. A room was arranged with three haptic workstations. Pre- and post-testing stations were on opposite ends of the room. Chaperones from SCI-VIS assisted with the testing stations and helped students who needed braille versions of the test.

Table 2 (Appendix) shows the demographics of the twenty students who tested at SCI-VIS. In summary, there were 12 males and eight females who ranged in vision loss from mild to total loss, from 10 to 17 years of age, and from fifth to twelfth grade. 

Data Analysis Procedures

Question 1: To what extent does the software function in the manner it was designed to function?
To answer this question, a modified heuristic usability survey was used with the adult and student experts. The usability survey consisted of items requiring a poor to excellent rating and open-ended questions. Similar instruments were used in studies testing virtual education products for the National Science Foundation’s Advancing Content through Interactive Virtual Environments project (Darrah 2012) and the US Department of Education’s Virtual Physics Lab project for both college and high school. For all instruments, the developers indicated that the feedback results provided them with valuable input and helped to eliminate many usability issues before the software was used in a classroom setting.

Survey results, numbers, and percentages of responses from rated items and the Repeating Ideas:Themes:Conclusions/Recommendations reporting of open-ended responses were given to the developers. Observation and interview data was summarized and given to the development team. For each app, all suggestions were summarized in a spreadsheet, which included suggested changes, planned actions, and the date the action was implemented in the software. Apps were revised based on all feedback given during testing with experts and testing in the classroom.

Question 2: Do the haptic-based apps adequately promote learning of science content from national science and mathematics standards to students who are blind or visually impaired?
To address this question, scores from the pre- and post-quizzes developed for three of the computer applications were used in the classroom setting and in the informal setting. A benefit of this design is the inclusion of a pre-test to determine baseline scores. To assure content validity the quiz content was reviewed by both the science teacher and the developer, and a table of specifications was developed to map the questions to the app’s learning objectives. In addition, each objective type test question contained an “I don’t know” response. Yu (2012) explains why this is important in the following:

Low reliability is less detrimental to the performance pretest. In the pretest where subjects are not exposed to the treatment and thus are unfamiliar with the subject matter, a low reliability caused by random guessing is expected. One easy way to overcome this problem is to include "I don't know" in multiple choices.  In an experimental setting where students' responses would not affect their final grades, the experimenter should explicitly instruct students to choose "I don't know" instead of making a guess if they really don't know the answer. Low reliability is a signal of high measurement error, which reflects a gap between what students actually know and what scores they receive. The choice "I don't know" can help in closing this gap. (para. 42)

Since the student sample for Phase I classroom testing was so small, no item analysis could be completed on the quizzes. Demographic data was collected from students pertaining to sex, age, grade, and level of vision. The pre- and post-tests were all scored by one person using a pre-defined rubric to insure consistency.

Question 3: Can the haptic-based apps be effectively deployed by classroom teachers?
To answer this question, the apps were used by the science teacher at a residential, academic school for the blind with three of her science classes. Observations of the apps being used by students in a classroom setting were conducted by the principal investigator and the developer. An observation protocol, focusing on feasibility, was used during the observation. In addition, the classroom teacher completed a feasibility survey consisting of a series of three open-ended questions. The reporting of open-ended responses was given to the developers.

Results of Testing

The results of the expert usability testing, classroom feasibility and student testing, and informal setting student testing are summarized below.

Expert Usability Testing

Many themes emerged from the usability testing with adult and student experts. Each app is different, so each of the apps scored differently in the ratings. However, some major themes emerged. The adult experts reported that the haptic-based apps were easy to use, keyboard keys were easy to adapt to and felt natural, audio instructions were helpful, time to learn to use the device was appropriate, color choices were appropriate, the content was accurate, and the apps were suitable and provided accessibility for use with students who are blind or visually impaired. Some negatives that were noted were the ease of accessing help during the app, the ability to set up the device without assistance, and the guidance given during the app to keep students from making a mistake. Many helpful open-ended comments were given by the reviewers. All comments were taken back to the developers and used to improve the app before it was released in the next version.

The student experts were asked many of the same questions as the adults, but they were also asked about how the app held their interest and helped them to learn. The student experts reported that the haptic device was easy to learn to use, the haptic-based apps were easy to navigate, the key strokes were easy to do, the audio was easy to understand, the apps held their attention, the apps would help them learn, the self-checks gave them feedback on their understanding, and the apps would be good for students who are blind or visually impaired. Some negatives that were noted from both the students’ comments and the observations of the students using the device included color choices on the screen, natural reader voices and pronunciation, using the buttons on the grip of the device, and the need for additional instructions. All comments were taken back to the developers and used to improve the app before it was released in the next version.

Classroom Feasibility Testing

Three classes at the West Virginia School for the Blind were observed using the haptic devices and the software by the researcher and the software developer. The same science teacher conducted all three classes. The teacher had downloaded and tested the apps on the machines before the students were scheduled to use them. She had only a few minor problems downloading the software and getting the device attached. The teacher gave the pre-test for the apps the morning before the students were going to work with them and gave the post-test for the apps the day after the students had worked with them. Each student had their own haptic device and computer with the haptic-based app software loaded. The observers did not interact with the students until after the class was over, when one observer briefly interviewed the students about their experience. The interview questions and responses are in the Classroom Usability Results below. The teacher was also given a short open-ended question survey by email several weeks after conducting the classes and these results are summarized below in the section called Teacher  Survey Results.  

Class 1: Three students were present, the grade level was seventh and eighth grades, and the subject of the class was science. During the class period the students did both the Three Dimensional Shapes App to introduce them to the device and the Exploring the Atom App. The teacher gave initial instructions and told the students to work at their own pace. The teacher circulated and answered questions about how to use the device and the software. While the students were interacting with the Exploring the Atom App, the teacher stopped the students and gave instructions and reinforced concepts presented in the app. The teacher also used the atom model in the app to reinforce other concepts she had already taught.

The students need to use JAWS or another screen reader to find the icon of the app on their computer desktop. During class they interacted with the software and raised their hands if they had a question. The students interacted with the software easily and were able to navigate using the keys given in the verbal instructions. The students did not have any issues getting used to the device or utilizing the keyboard for navigating the software. They were engaged the entire class period. The students wanted to know how much the device cost and were eager to give suggestions for other apps that could be developed.

Class 2: Two students were present, the grade level was seventh, and the subject of the class was science. The students had some cognitive disabilities and one student quit working on anything right after the beginning of the class (the teacher indicated that this was normal for her). During the class period the students did both the Three Dimensional Shapes App to introduce them to the device and the Exploring the Atom App. The teacher gave initial instructions and told the students to work at their own pace. The teacher circulated and answered questions about how to use the device and the software. In this class she spent more time helping them understand the basic shapes and textures. The teacher sometimes guided their hands to help them feel the objects. After the one student stopped working with the software, the teacher focused on the other student and sat with her giving her personal instruction. 

Even though the students in Class 2 needed more individualized help, they still knew the keyboard well and were able to navigate the software using keyboard strokes. The student who chose to participate was highly engaged and was able to utilize the device properly and effectively. This student said it was awesome, but she didn’t understand all of the vocabulary used in the app.

Class 3: Two students were present, the grade level was sixth, and the subject of the class was science. During the class period the students did both the Three Dimensional Shapes App to introduce them to the device and the Exploring the Atom App. The teacher gave personal instructions to a student who is blind. The teacher spent some time orienting her to the device. The other student in the class had low vision. He didn’t need as much assistance and was working independently.  

The students were intrigued by the device and software. After just a few scenes the students got used to the device and how to use it. The student who was blind needed more initial help understanding the scene and how to get around in the app, but quickly became independent using the software.

Student Comments: After the classes were over the students in all three classes were asked a few questions. All students said that the technology helped them understand the content better. All students said that the activity was fun. All students said that the activity was easy to do. Some of the comments included “Awesome”, “Liked the textures” and “It was fun”.

Teacher Survey Results

After testing the device and apps in the classroom, the teacher was given a short survey to collect her impressions of how the implementation went. Below is a list of four open-ended questions that the teacher was asked after using the device and apps with her students in the classroom. Both the questions and her responses are listed below.

  • What worked and what didn't work?
    “When not working one on one with the student, I could not interject additional information because I couldn't hear when a question was being asked so I wasn't interrupting directions. On a positive note, all directions and questions could be repeated.”
  • What would you do differently?
    “If more than one Falcon is in use, set up the apps so they can be used individually or as a group where there are questions throughout the students can solve together and then a questionnaire at the very end so the student and teacher can assess themselves, and not always as a whole group. Then a screen can pop up and say con-grads you answered 3 out of 4 correct, or nice try, or you need more practice.”
  • How did the addition of the apps activity impact student learning?
    “They made the topic more interesting with a hands-on activity to engage the student(s) on an otherwise difficult topic to teach student(s) with limited spatial skills.”
  • How did the integration of the apps change the way you normally teach the lesson?
    “It can be integrated in several ways to keep lessons fresh. They can be used as an introduction, a supporting activity, informative lesson, or an assessment of what the student has gained on the topic as a whole. I would like to use it in all locations, varying per topic. Most likely after the material and vocabulary have been introduced.

Results of Student Learning Testing

As mentioned previously, at the West Virginia School for the Blind the classes are very small.  This resulted in four sets of matched pre-test and post-test scores for the app tested. Since the topic of the Three Dimensional Shapes app was mainly used to familiarize the students with the haptic device and software, a pre-test and post-test for this app was not appropriate for this age level. The Exploring the Atom app was grade level content and more challenging for the students. The pre-test and post-test was composed of seven questions with a minimum possible score of 0 and a maximum of 14. Respondents (N = 4) mean pre-test score was 4.75 (SD = 4.99) and mean post-test score was 9 (SD = 4.76). A nonparametric Wilcoxon Signed Ranks Test for related samples was computed. The distributions were significantly different (α = 0.05). It appears reasonable to attribute this increase from pre-test to post-test score to use of the app, since the app was being used as a stand-alone lesson, not in the context of any additional lesson on the topic.

Three apps were tested in the informal setting of a science camp for students. Students showed significant learning gains on all three apps (Appendix, Table 3). For the Exploring the Atom app, the pre-test and post-test were composed of seven open-ended questions with a minimum possible score of 0 and a maximum of 14. Respondents (N = 14) mean pre-test score was 2.21 (SD = 3.93) and mean post-test score was 5.36 (SD = 4.67). A paired t-test was used to show there was a significant increase between the pre-test and post-test scores (p = 0.002). For the Gravity on Planets app, the pre-test and post-test was composed of seven open-ended questions with a minimum possible score of 0 and a maximum of 16. Respondents (N = 9) mean pre-test score was 5.67 (SD = 3.81) and mean post-test score was 8.22 (SD = 3.63). A paired t-test was used to show there was a significant increase between the pre-test and post-test scores (p = 0.013). For the Surface Area of a Cube app, the pre-test and post-test was composed of five open-ended questions with a minimum possible score of 0 and a maximum of 11. Respondents (N = 14) mean pre-test score was 1.36 (SD = 0.74) and mean post-test score was 3.43 (SD = 2.90). A paired t-test was used to show there was a significant increase between the pre-test and post-test scores (p = 0.006). In each case, it appears reasonable to attribute this increase from pre-test to post-test scores to use of the apps, since the apps were used as a stand-alone lesson in an informal setting with no teacher intervention.

Analysis of results

The preliminary results show promise for the haptic-based apps as valuable classroom materials for middle schools students who are blind or visually impaired. The adult experts (many of whom are teachers) and the classroom teacher seem to agree that the haptic device and the haptic-based materials will be usable in a classroom setting and will be usable by students who are blind or visually impaired. Usability and feasibility testing also revealed that the haptic device is easy to set up and learn to use. Further, this phase of testing revealed and helped solidify best practices for developing the apps for the target audience. The preliminary student testing, although with small numbers, has shown the promise of learning occurring with the use of these apps. 

Conclusions

Haptic devices and haptic-based software can be a useful tool in the classroom. This technology can be used by teachers along with the already existing techniques of tactile graphics, 3D models, and description in the math and science classroom to bring benefits to students. This innovative technology and software can integrate rich information through the sense of touch to add to the information being provided through auditory and visual means. This can be a new tool for teachers and students to add to their tool box. 

Implications for Practitioners and Families

The haptic device and haptic-based software described in this article provide stand-alone lessons on math and science that engage students through an exciting new technology. The software incorporates 2D and 3D touch, audio, and high-contrast graphics to provide a truly multi-sensory learning experience. Through these materials, which are aimed at grades five through nine, students can interact with a virtual world. For example, they can feel 3D shapes related to the parts of a plant cell or feel the difference between the gravity on Earth and the gravity on Mars. Students find the software and the device enjoyable and easy to use. Teachers find the software and device easy to implement in the classroom. Parents could also use this at home with their children to supplement the content they receive at school. 

Appendix

Table 1 – Developed Apps

Apps

Purpose

1. Three Dimensional Shapes

Learning to use the haptic device and about textures and shapes

2. Exploring the Atom

Learning about the parts of an atom and about three different elements

3. Surface Area of a Cube

Learning to compute the surface area of a cube

4. Gravity on the Planets

Learning about mass and gravity on the terrestrial planets

5. Volume of a Cube

Learning to compute the volume of a cube

6. Gravity Holds It All Together

Learning about the gravity within the universe and how it affects celestial bodies

7. Exploring the Plant Cell

Learning the parts of a plant cell

8. Surface Area of a Cylinder

Learning to compute the surface area of a cylinder

9. Volcanoes

Learning how volcanoes form

10. Blood Cells

Learning the types of blood cells and the types of blood

11. Graphing Lines

Learning to model a problem with a linear function

12. Our Solar System

Learning about the planets in our solar system

13. Coral Reef Ecosystem

Learning about the members in the coral reef ecosystem

14. Circulatory System

Learning about the parts of the circulatory system

15. Volume of 3D Shapes

Learning to compute the volume of a cylinder, cone, and sphere

16. Symmetry of Shapes

Learning about line symmetry of shapes

17. Desert Ecosystem

Learning about the members in the desert ecosystem

18. Mean, Median, Mode

Learning to compute mean, median, and mode

19. Changing Landscapes

Learning about natural and man-made occurrences that cause changes to the landscape

20. Forces of Drag

Learning about drag forces and what effects these forces

Table 2 – Student Demographics

This table summarizes the demographics for the students who were tested in the informal setting at SCI-VIS. There were 12 males and 8 females who ranged in vision loss from mild loss to total loss. Their ages ranged from 10 to 17 years old and they ranged from fifth to twelfth grades.

Sex

# of Students

Vision Level

# of Students

Age

# of Students

Grade

# of Students

Male

12

Mild

7

10 years

2

5th

3

Female

8

Moderate

4

11 years

4

6th

4

 

 

Severe

4

12 years

1

7th

2

 

 

Profound

1

13 years

5

8th

6

 

 

Near total

2

14 years

4

9th

2

 

 

Total loss

2

15 years

2

10th

2

 

 

 

 

16 years

1

11th

0

 

 

 

 

17 years

1

12th

1

Table 3 – Compiled Test Scores

This table summarizes pre-test and post-test means, standard deviations, and p-scores from the paired t-test. The scores show significant learning gains for all three apps.

 

Exploring the Atom
(n = 14)

Gravity on the Planets
(n = 9)

Surface Area of a Cube (n = 14)

Pre-Test Mean

2.21

5.67

1.36

Pre-Test Standard Deviation

3.93

3.81

0.74

Post-Test Mean

5.36

8.22

3.43

Post-Test Standard Deviation

4.67

3.63

2.90

p-value

0.002

0.013

0.006

References

Brewster, S.A. (2002). Visualization tools for blind people using multiple modalities. Disability and Rehabilitation, 24(11-12), 613-621.

Brooks, F. P., Ouh-Young, M., Batter, J. J., & Kilpatrick, P. J. (1990). Project GROPE: Haptic displays for scientific visualization. ACM Computer Graphics, 24(4), 177–185.

Burkman, E. (1987). Factors affecting utilization. In R. M. Gagne (Ed.) Instructional Technology: Foundations (pp. 429–455). Hillsdale, NJ: Lawrence Erlbaum Associates.

Clark, C., & Jorde, D. (2004). Helping students revise disruptive experimentally supported ideas about thermodynamics and tactile models: Computer visualizations and tactile models. Journal of Research in Science Teaching, 41(1), 1–23.

Darrah, M. (2012). The use of touch technology in science to provide access for students who are visually impaired. Journal of Technology Integration in the Classroom, 4(1), 45-50.

Ferrell, K. A. (2007). Issues in the field of blindness and low vision. National Center for Low Incidence Disabilities, University of Northern Colorado. Retrieved from http://www.unco.edu/ncssd/resources/issues_bvi.shtml

Jones, M. G., Andre, T., Superfine, R., & Taylor, R. (2003). Learning at the nanoscale: The impact of students' use of remote microscopy on concepts of viruses, scale, and microscopy. Journal of Research in Science Teaching, 40(3), 303-322.

McGookin, D. K., & Brewster, S. A. (2007). Graph builder: Constructing non-visual visualizations. In Bryan-Kinns, N., Blandford, A., Curzon, P. and Nigay, L. (Eds.) People and computers XX – Engage: proceedings of HCI 2006 (pp. 263-278). London: Springer.

Minogue, J., & Jones, M. G. (2006). Haptics in education: Exploring an untapped sensory modality. Review of Educational Research, 76(3), 317-348.

National Federation of the Blind. (2011). Blindness statistics. Retrieved from https://nfb.org/blindness-statistics

Shannon, T. D. (1973). U.S. Patent No. 3780225 A. Washington, DC: U.S. Patent and Trademark Office. Retrieved from https://www.google.com/patents/US3780225?dq=US3780225&hl=en&sa=X&ei=9FkrUr6kOqroiwLqzoGYBA&ved=0CDQQ6AEwAA

Sjostrom, C. (2001). Using haptics in computer interfaces for blind people. In M. M. Tremaine (Chair) CHI '01 Extended Abstracts on Human Factors in Computing Systems (245-246). New York: ACM Press.

Smith, A.J., Geruschat, D., & Huebner, K. (2004). Policy to practice: Teachers' and administrators' views on curricular access by students with low vision. Journal of Blindness & Visual Impairment, 98(10), 612-628.

U.S. Department of Education. (2012). 31st Annual Report to Congress on the Implementation of the Individuals with Disabilities Education Act, 2009. Office of Special Education and Rehabilitation Services. Retrieved from http://www2.ed.gov/about/reports/annual/osep/2009/parts-b-c/index.html 

Yu, C. H. (2012). Reliability and validity. In Creative Wisdom [Website]. Retrieved from http://www.creative-wisdom.com/teaching/assessment/reliability.html

Yu, W., Reid, D., & Brewster, S.A. (2002). Web-based multimodal graphs for visually impaired people. In Keates, S., (Ed.) 1st Cambridge Workshop on Universal Access and Assistive Technology (CWUAAT), Cambridge, England. Retrieved from http://eprints.gla.ac.uk/3228/

Williams, R. L., Chen, M., & Seaton, J. M. (2003). Haptics-augmented simple-machine educational tools. Journal of Science Education and Technology, 12(1), 1-12.


The Journal of Blindness Innovation and Research is copyright (c) 2014 to the National Federation of the Blind.