Future Reflections Special Issue: Technology
What's the Score?
Issues in Standardized Testing for Blind Students
by Pat Renfranz
From the Editor: After earning her PhD in biology, Pat Renfranz worked as a scientist and university instructor. Currently she works at home, and she is a vocal advocate for blind children. She serves as vice president of the Utah Parents of Blind Children (UPBC) and as treasurer of the National Organization of Parents of Blind Children (NOPBC).
"I took a multiple choice test today, and I don't think the answer was one of the choices," Caroline, my blind, fourth-grade daughter, remarked a number of years ago.
My daughter's comment raised questions that launched my investigation into standardized testing and how it is managed for blind students. I learned about state- and district-mandated testing and how the tests were accommodated (or not) in order for my daughter to take them. Together with Utah Parents of Blind Children, I discovered a great number of problems in Utah, such as a lack of proofreading for tests transcribed into Braille. These problems led Utah Parents of Blind Children and the NFB of Utah to file a complaint against the Utah State Office of Education in 2009.
By writing this article, I hope I can help you navigate the rapidly changing world of testing. Perhaps your student or child can avoid many of the problems Caroline has faced. I will focus on end-of-year testing that is meant to measure a school's Adequate Yearly Progress, as mandated by federal education rules. Typically known as Criterion-Referenced Tests (CRTs), these tests measure what a student knows at the end of the school year and are based on the state's educational curriculum guidelines. I will describe problems we have faced, accommodations for blind students, and what you can do to ensure that your child's or student's tests are accessible and appropriately produced. I will also discuss what lies on the testing horizon, including computer-based testing and computer-adaptive testing.
In this article I will not address the critically important matter of alternate assessments. With the approval of the IEP team, these end-of-year tests are given to students with significant cognitive disabilities. In 2003 the US Department of Education issued regulations allowing states to develop alternate standards and assessments for students with the most significant cognitive disabilities, thus exempting those students from the regular end-of-year tests given by that state. The decision about whether or not a student should have an alternate assessment, as well as the design of that assessment, are important and challenging issues for parents with children who have significant cognitive disabilities in addition to visual impairment. This topic fully deserves its own article.
Why I Am Interested in High-stakes, End-of-year Tests
Over the years Caroline has dealt with tests that were not ordered in the appropriate format (Braille), tests arriving late, answer sheets with no reasonable way for her to mark the answers, tests given before material was covered in class, tests with Braille transcription errors that made questions unanswerable, and lost test results. It has proven very difficult to identify the source of these problems, as parents and teachers have limited access to these high-stakes tests prior to and after they are given. My child was my only avenue for identifying problems on these tests. Her confidence in being able to answer a particular test question and to recognize unfairness in test administration were key as I tried to improve her situation. I also took notice any time her test scores did not reflect her capabilities.
Why should we care about these tests? These federally mandated tests are given in part to hold local and state educational programs accountable (the Adequate Yearly Progress measure). Furthermore, a good test should show whether a student is making grade-level progress in school. Theoretically, it should show how blind students perform compared to their sighted peers. In some states, these tests are required for high school graduation, thus potentially having a huge impact on a student's future.
Should blind students be included in statewide testing? The Individuals with Disabilities Education Act (IDEA) states that all children with disabilities are required to participate in statewide and district assessment programs. Their participation can be with or without accommodations. An alternate assessment can be provided if the IEP team determines that it is appropriate. States are required to report results, such as how many students with an IEP participate in the assessment and how they perform. District-wide and formative assessments should be accessible, too.
Accommodations: Testing Ability, Not Disability
Accommodations are changes to testing materials, procedures, or setting in order to prevent a student's disability from interfering with test results. In other words, accommodations facilitate how the test is administered and responded to. Appropriate accommodations reduce error in test scores due to poor access. They do not change or replace the skills that the test is designed to measure. Each state develops its own set of acceptable accommodations. For an excellent discussion of accommodations on statewide assessments, visit <http://nichcy.org/research/ee/assessment-accommodations>.
Who decides who will use which accommodations? Accommodations that will allow a blind student to participate in statewide testing are selected by a student's IEP team and are thus individualized. Therefore, not all blind students in a given state necessarily receive the same accommodations. The IEP team must understand the format of the test and what specific skills are being measured. The team must know the student's abilities and the instructional accommodations that he/she receives. The student must be familiar with the use of these accommodations before the test is given. Testing accommodations should be documented in a prescribed section of the IEP; they are not provided as a favor.
Categories of accommodations include presentation, response, setting, and timing. For blind students, accommodations might include the following:
1. Presentation: Braille (contracted or uncontracted, literary and Nemeth Code); tactile graphics; large print or magnification; audio/speech assistance.
2. Response: Oral dictation of answers to a scribe, marking answers in test booklet, marking on special answer sheet, Braille transcription. Be aware that transfer of answers to a bubble sheet or within a computer-based test must be done carefully, and that the student's original responses (for example, answers in the test booklet) should be kept securely for a given period of time.
3. Setting: Test taking in another room and/or with ample space, appropriate lighting.
4. Timing and Scheduling: Extended time, multiple sessions to help deal with fatigue. (If a student needs an inordinate amount of time to complete the test, it is probably worth requesting an assessment of the student's reading medium and asking whether the amount of training in the use of accommodations needs to be increased on the IEP.)
5. Special Tools: Tactile or large print ruler or protractor, talking calculator, Braille or large print scientific tables, abacus, marker, magnification device, manual Brailler with paper, special paper, Braille notetaker.
Keep in mind that an allowable accommodation does not necessarily equal an appropriate accommodation. For example, instead of providing a reading assessment in Braille, your school may wish to provide audio output or a live reader. The school might insist that the student will perform better that way, especially if he/she is just starting to learn Braille. I would suggest that your child or student take any language arts assessments at grade level in the format in which she or he is learning to read.
Make the IEP team take the time to study accommodations. At the end of a long, stressful meeting, it's easy to gloss over this section. The accommodations a student needs may change with time and may require additional IEP goals.
Special Considerations
1. The IEP team needs to plan for accommodations long before tests are given. Tests in a special format such as Braille must be ordered early so that there is sufficient time to prepare them and have them proofread.
2. Ask whether the producer of the test uses recommended formatting and whether a certified Braille transcriber will be involved. It is also worth finding out if the tests are proofread.
3. We have found it essential to ensure that a print test accompanies any Braille materials, so that a sighted test administrator can address any questions the student may have.
4. If the test is not proofread, you might request that a teacher of the visually impaired review the test for errors. Be aware, though, that because these tests are high stakes for schools, security is tight; schools, districts, and state offices of education may be reluctant to allow anyone to preview the test.
5. Your student should take tests at the same time as classroom peers.
6. Ask how tactile graphics will be produced or whether they will be omitted. The student should be provided with sample graphics in order to practice using that particular medium.
7. If other students are able to take practice tests, your child or student should be able to take them as well.
Caveat: Some blind students are being provided accommodations under Section 504 of the Rehabilitation Act rather than under IDEA. If you find that your 504 child or student requires extensive training in order to utilize accommodations, then he/she should be evaluated for special education services under an IEP.
Big Changes on the Horizon: or, What Is This Article Doing in an Issue on Technology?
Across the nation, the preferred testing format is moving from paper-based to computer-based tests. Computer-based tests (CBTs) are administered on a computer; students use a computer to access and answer the test questions. There are no test booklets or bubble sheets. CBTs are administered and graded efficiently, and many students prefer them. Furthermore, this format has the potential for built-in accommodations, such as speech output and highlighting. Sometimes a paper version of the test is also available. Currently, at least twenty-six states give a statewide test via computer, and sixteen of them offer an end-of-year CRT assessment on computer as well.
The use of computer-based tests will become widespread with the adoption of the Common Core State Standards Initiative. (For more information on this initiative, visit <www.corestandards.org>. Teachers, school administrators, and experts collaborated to develop the Common Core standards for grades K-12 in English language arts and mathematics. The Common Core aims to make educational standards more rigorous, better preparing students for postsecondary pursuits, no matter what school they attend. While the Common Core is not a federal government initiative, forty-five states, the District of Columbia, and two US territories have adopted the standards and are currently changing their curricula to meet them. The states that have not adopted the Common Core are Texas, Virginia, Alaska, Minnesota, and Nebraska.
Okay, how does the Common Core affect testing? Current federal education law (No Child Left Behind) mandates that every state must give grade-level Criterion-Referenced Tests (CRTs) to determine students' level of proficiency. These tests are designed to match each state's educational standards. With the current system, in which every state adopts its own educational standards, there are fifty sets of standards and fifty criteria for proficiency. Thus, it is not currently possible to compare student proficiency between states using data from CRTs.
Since the Common Core standards will be shared between states, CRTs given by those states must be changed so that they align with the Common Core. The design of new assessments to align with the Common Core is stimulating the development of new computer-based testing protocols. Two large, multi-state consortia have received significant grants to develop computer-based assessments. These new assessments are scheduled to be implemented in the 2014-15 school year in states that have adopted the Common Core. Funding has also been awarded to develop new alternate assessments.
If you wish to learn more about the multi-state consortia or to find out which consortium your state belongs to, visit one of these websites: Partnership for Assessment of Readiness for College and Careers, <www.parcconline.org> or the Smarter Balanced Assessment Consortium at <www.smarterbalanced.org>.
The Smarter Balanced consortium is developing a specific type of CBT called a Computer-Adaptive Test, which is thought to give more accurate results than a regular CRT. A Computer-Adaptive Test adapts to the student's level as the student is taking the test. In other words, the question a student is given depends on whether his previous answer was correct or incorrect. It is possible that no two students will take exactly the same test. Other adaptive test products are already available, such as the NWEA's Measures of Academic Progress tests and the Scholastic Reading Inventory.
My daughter's experience with computer-adaptive tests has made me greatly concerned about their accessibility. Our school district uses the Scholastic Reading Inventory to measure students' reading levels. This test has over five thousand possible questions in its database; thus it cannot be provided in embossed Braille. Our district appears utterly unprepared to give the test to a Braille reader. Furthermore, while speech output is ostensibly embedded, no one has figured out how to use it. In any case, the use of speech begs the question of whether speech is an appropriate accommodation on a reading test.
Planned Accommodations for CBTs
Test developers plan to embed accommodations into CBTs by utilizing the principle of universal design. These accommodations include magnification, highlighting, contrast, and audio/speech. Test developers also anticipate the use of refreshable Braille displays. Furthermore, if a paper test can be produced, then any of the standard accommodations such as embossed Braille, large print, and tactile graphics can theoretically be used.
Test developers must take into account special requirements for the accessibility of mathematics and graphics. Engineers outside the testing industry have developed software to allow math composed in certain computer languages, such as MathML, to have output in Nemeth Braille code. Examples of this software are liblouis, the Tiger Software Suite, and BrailleBlaster. Presumably, the output could be embossed or read on a refreshable Braille display. The Smarter Balanced consortium has produced guidelines for what it terms "tactile accessibility," including output in literary Braille, Nemeth code, tactile graphics, and physical manipulatives. The implication from the guidelines is that questions tagged as requiring a tactile output could be produced on paper or on a Braille display. Incorporating Nemeth Code and tactile graphics accessibility into these tests will require a great deal of collaboration between test developers and adaptive software developers. How tactile graphics and/or embossed Braille can be prepared for use on a computer-adaptive test, a test for which the questions a student might receive are not predictable, will require extensive study.
Many challenging questions lie ahead of us. What method will determine the best way to present accommodations for blind students? Who will give blind students the training necessary to take computer-based tests successfully? Who will train teachers of blind students currently in the field? Is it reasonable to expect that all blind children will be technically adept enough to maneuver through a test using a keyboard and/or a refreshable Braille display?
What Parents and Educators of Blind Students Can Do
1. With either current test protocols or the new computer-based tests, it is important for parents and teachers to be active members of the IEP team. Make sure that the accommodations your child or student requires are fully documented on the IEP, and be sure everyone on the team understands that test accommodations must be used in the regular school setting.
2. Know who your school, district, and state testing administrators are, in both general education and special education. District and state administrators need to know that these tests are important to you and your child or student. My state has an assessment specialist in special education who has been very helpful in working out particular problems.
3. Ask the school, district, and state education authorities what tests are given to students in your child's educational setting, and find out about the testing window and format of those tests. In my state, the CRTs are given on computer, and our district also gives a number of computer-based tests throughout the year to help teachers monitor student progress. Currently, Utah's computer-based CRTs can be provided on paper (Braille or large print), but the district tests, including Acuity, Scholastic Reading Inventory, and Yearly Progress Pro, apparently are not accessible to Braille readers.
4. Ask state assessment officials if and when computer-based tests will be given to students in your state. Ask specifically how blind students will be accommodated on those tests. With the advent of computer-based tests, you must anticipate the technology needs of your child or student today, so that she/he has access to the technology and training on how to use it before the test day arrives.
5. Talk to your child about which tests she is being given or being excused from taking, and find out what her experience has been. Request your student's scores and question discrepancies between results and the student's typical performance. Just this past month, I noticed that Caroline's posted proficiency score on a Biology CRT was unexpectedly low. I contacted an administrator in the school district, who worked with the state office of education to discover that her test had been scored using an Earth Science CRT key. Her test was rescored, and the correct results were posted.
6. If you suspect errors or a lack of accessibility, let administrators know as soon as possible, and in writing. Request that embossed or large-print test booklets be saved. Remind school administrators of the US Department of Education Office of Civil Rights "Dear Colleague Letter" of May 26, 2011, concerning the use of emerging technologies in the classroom. If the problems seem irreparable or the response seems inadequate, consider filing a complaint with the Office of Civil Rights or under IDEA.
7. Do not keep test issues to yourself. If your blind child faces poorly prepared tests or is not adequately prepared to use available accommodations, chances are that other blind students in your state are dealing with the same problems. Sharing personal examples of problems allows the NFB and NOPBC, as well as state POBC chapters, to advocate effectively for all blind students. Send a copy of your letter or email to your local Parents of Blind Children chapter and to the president of your state's NFB affiliate. Feel free to send me a copy, too. I will see that it gets attention from NOPBC and the NFB's national office.
To ensure that test developers are aware of the needs of blind students before new tests and software are designed, the NFB has sent letters of concern to leaders of the state consortia, both for typical assessments and alternate assessments. The NFB has had discussions regarding these issues with high-level Department of Education officials. If we, as parents and teachers, are vigilant at the ground level, then high-stakes, end-of-year tests are more likely to be accessible and appropriate for our children.
Please let me know if I can be of any assistance to you. I can be reached by email at [email protected].