From the Editor: For several months now we have been hearing rumors that the Apple iPhone is actually accessible or soon will be. Tony Olivero is an access technology specialist in the Jernigan Institute International Braille and Technology Center. Here is his report of learning to use an iPhone:
Apple computer is generally regarded as a company on the cutting edge of consumer computing technology. Many of Apple’s products have brought advanced ideas into the mainstream and popularized sleek form and easy-to-use interfaces. The iPhone was certainly no exception. This device was the most powerful handheld computing, telephony, and information-access device on the market at the time of its initial launch. Unfortunately, as happens with many new pieces of consumer technology, for two years blind users were denied access to this cutting-edge product.
This lack of access changed in June 2009 when Apple released the iPhone 3GS, the latest version of the phone’s hardware platform with the ability to use third-generation mobile phone networks to provide high-speed data connectivity to the device. The iPhone 3GS is also the first touch-screen device to provide out-of-the-box, built-in screen access. Incorporating a version of Voiceover (Apple’s screen-access product included in the Macintosh Operating System since version 10.4 Tiger) the iPhone provides blind users access to built-in software applications, including email, text messaging, the Safari Web browser, and the telephone itself. While this article will primarily focus on the speech output provided by VoiceOver, accessibility features called “Zooms” and “White on Black” may be useful to low-vision users.
The iPhone 3GS comes in either a black or white case with a silver Apple logo emblazoned on the back and offers either 16 or 32 gigabytes of storage. The phone is a candy-bar-style device, which means it is flat and does not fold like a flip phone. The back of the device is convex, allowing the phone to rest comfortably in the user’s hand. Users interact with the iPhone almost entirely by using a touch screen. The phone has only four buttons.
Most of the front surface is the screen, where information is displayed to sighted users and they control the phone. At the top of the front panel is the speaker hole, where sound is heard when the phone is held to the ear. At the bottom is a round button, the home key. When pressed, this key returns the user to the home screen, from which applications are chosen.
Along the left edge is an up-and-down rocker switch used to control volume for system sounds, speech output, and callers. Just above the volume control is a two-way toggle switch used to select silent mode or sound. When the toggle is nearer the front panel, the silent mode is disabled. Options for silent mode can be customized by using the Sounds tab of the Settings application.
The headphone jack is located at the top edge of the phone (it accepts a standard 3.5mm plug for stereo headphones with or without a built-in microphone). The Screen Lock/Power button is also located on the top edge, at the right side. Pressing this button once turns the screen off, placing the phone in locked mode, or wakes it up, presenting the user an unlocked screen. Double-tapping the Unlock button provides access to the phone. To power the phone off, hold down the Power button for ten seconds and, when prompted, double-tap the Power button. The bottom edge of the phone contains no controls, but it does house the speaker and dock connector, where the charging/data transfer cable is inserted.
In addition to the phone, users receive a CD containing iTunes software; a USB cable that connects to the dock connector; a pair of stereo ear buds with a microphone, volume control, and Answer/Hang-up button built into the right-hand ear bud cord; an AC adapter that accepts the USB cable and allows charging from a standard outlet; and printed documentation.
In order to set up the iPhone for the first time, users must connect it to a computer running iTunes. When iTunes detects the phone, it will prompt the user to verify that he or she is the account holder and finalize the configuration of the phone. Once the phone has been authorized, the user can move onto configuring options and establishing synchronization parameters.
When iTunes recognizes that an Apple device has been connected to the computer, a devices node appears in the sources tree. Under this node will be another node for each connected device. To configure the option for the iPhone, use the F6 key to move to the Sources Tree, locate your iPhone with the Up and Down Arrows, and press the F6 key (usually twice) until you move into the iPhone pane.
The content pane for the iPhone contains tabs allowing you to customize which, if any, content from iTunes will automatically be synchronized when the iPhone is connected. You can instruct the computer to copy certain playlists, podcasts, videos, and other media items; backup installed applications; and synchronize mail, contacts, and calendar appointments from your mail program, among other options. One of the most important options on the General tab to blind users is the Universal Access button. Activating this button allows the user to turn on one of the iPhone’s accessibility features, which include VoiceOver (providing speech output), Zooms (providing magnification), and White on Black (providing reverse contrast video).
iTunes is also supposed to detect whether screen-access software is running and enable VoiceOver by default. Owing to a SIM card failure requiring a trip to an AT&T store, I was unable to test this feature because I had the AT&T representative enable VoiceOver for me (on the plus side, however, he did know exactly what VoiceOver was).
Users can also enable or disable accessibility options by going to the Settings icon on the home screen, choosing General, and then selecting the Accessibility button. Finally, the iPhone OS version 3.1, released on September 9, 2009, allows the user to specify an accessibility feature to be enabled and disabled by triple-tapping the Home button. One item of note: since they use identical gestures, Voiceover and Zooms cannot be active at the same time.
Ninety-nine percent of user interaction with the iPhone is done on the touch screen. Sighted users tap icons representing everything from applications to letters on the keyboard and numbers on the telephone dial pad. This phone has absolutely no physical keyboard. Apple has created a set of gestures that, when used on the touch screen, assuming accessibility features have been enabled, allow a blind user to interact with the iPhone.
The most basic gesture is touching the screen. When a user taps anywhere on the display, the object under the finger will be identified. For example, touching the lower left corner of the home screen tells the user he or she is on the Phone application. The instruction follows to “double-tap to open.”
For the blind user the double-tap is the equivalent of the sighted user’s tap on the screen. This gesture activates the icon under the user’s finger. The double-tap can be used to open an application, open an email or text message, or enter text through the on-screen keyboard.
Users are not limited to tapping the screen to find icons. If one is not familiar with an application, a finger can be slid around the screen as well. Objects under the finger will be identified. In addition to the double-tap, the user can keep one finger on an icon while tapping the screen with a second finger. This performs a simulated double-tap and can be faster in some cases than performing the actual double-tap maneuver.
The flick is another technique for moving around applications. This gesture is performed by swiping a finger from left to right, right to left, up, or down on the screen. Performing a left-to-right or right-to-left flick moves the user between interface elements: icons, text areas, elements of a Web page, items in a list box, or any other screen component. The flick can be a useful way to explore a new application or move through a list quickly. It is possible to tap when one means to flick, thus losing the original cursor focus, but this is less of a problem with practice. The up and down flick gestures are used to move through elements by a specified increment set by the Rotor Control. For instance, if the Rotor is set to “character,” flicking down speaks the next character in the current element. This gesture can also be used to move the insertion point when editing text.
The Rotor control is Apple’s solution for giving the user a way of selecting a method for moving through the currently active control or interface area and is designed to emulate an analog dial on a radio. By placing two fingers on the screen and rotating one clockwise or counterclockwise, different options can be selected. The most basic of these are “character” and “word.” In an application supporting HTML navigation, such as the Safari Web browser or an email message written in HTML, the available options on the Rotor control increase to support navigating by headings; links; unvisited links; images; visited links; and, in the most recent iPhone software, static text. Copy and paste functionality is also available on the Rotor dial.
Other gestures are also available. Placing two fingers on the screen and swiping downward instructs VoiceOver to begin reading from the current position and stop when it reaches the bottom of the text or screen. Tapping two fingers on the screen silences speech. Placing three fingers on the screen and sliding them left or right moves one between tabs in multitabbed areas such as the home screen. To move to the next page, slide your fingers to the left (like turning pages in a book). Placing three fingers on the screen and sliding up or down scrolls through content that spans more than one vertical screen-full. Users should be aware that you slide your fingers down to move toward the top of the content and slide up to move toward the bottom. This may be confusing at first since the typical scroll bar on a computer is dragged down to reach the bottom and pushed up to find the top of the document.
The pinch gesture is accomplished by placing two fingers on the display and moving them closer together or further apart. Blind users use it to increase or compress the amount of selected text when performing a cut or copy function.
The final two gestures are used to control the VoiceOver speech and Screen Curtain functions. A three-fingered double-tap silences or enables speech output. A three-fingered triple-tap enables or disables the Screen Curtain function. As the name suggests, Screen Curtain blanks out the screen so nothing is visible to passersby.
Many people find text entry the most frustrating part of the iPhone experience. Sighted users are quite likely to share this opinion. When a user double taps to activate a text-entry field, the iPhone converts the bottom half of the screen to an onscreen keyboard. Users then select keys by using the tap, double-tap, or slide and tap methods. Some users have reported success using two thumbs to type, one thumb finding the letter on its half of the keyboard while the other is available for tapping duty once the letter is found. I have been most successful holding the phone in my left hand while my right index finger skims over the keyboard to the desired letter and my middle finger performs the tap to enter it.
Many applications allow the user to turn the phone sideways, activating landscape mode. This provides a wider keyboard horizontally with bigger keys, but it is compressed vertically.
With a combination of time, muscle memory, and practice, typing speed increases. It is also possible to adjust the keyboard echo verbosity to echo entered letters, words, or neither. I have found that turning off letter echo (letters are spoken when a finger is above them, but only a click is heard when they are entered) and allowing words to be echoed provides a nice balance between speed and editing as I type. The built-in predictive-text features of the iPhone also provide for corrections if you press the wrong letter. For example, typing “thr” and pressing space will generally result in the word being corrected to “the.”
One thing to remember, and something that confused me at first, is the direction of entered and deleted text when editing. Using the Rotor control, you can elect to move by character or word to edit your text. The up and down flicks then move you through the text by the selected element. Remember that, unless you are at the end of the text, the Delete key will delete in the direction you last flicked. So, if you are expecting to delete the character before the one just spoken and you have been moving forward, you must flick upward to orient the cursor to move back. Otherwise, pressing Delete deletes the most recently spoken character. One other cautionary note--occasionally edit fields on Web pages and in certain applications do not speak correctly when echoing characters or editing, but the text has been successfully entered.
This screen is the launch pad for all iPhone applications. When the round Home button is pressed, the user is taken to the first screen of applications. Up to nine pages can be used to display icons for installed programs. Each page consists of a four-by-four grid of icons and a single row of 4 icons at the bottom of the screen. The single row, which by default contains the Phone, Mail, Safari, and iPod apps, is present regardless of which page is currently being displayed. The three-fingered left and right scrolling gestures can be used to move between pages.
The Safari Web browser provides access to Web pages and some streaming media content. By default the Safari icon is the third from the left in the static bar at the bottom of any home screen. Web browsing using VoiceOver has generally been a pleasant experience. The Web browser provides an address bar at the top for directly accessing a URL, a text box to enter a search that will automatically display the results in Google, and bookmark features to allow rapid access to frequently used Websites.
When you are in the browser, several mechanisms exist to locate information and read content. By using the left and right flick gestures, it is possible to move through page elements sequentially. All elements will be spoken as you encounter them. The options available with the Rotor control increase to include frequently used HTML elements including headings, links, form controls, images, and text. By selecting one of these options and employing the up or down flick, you can move to the specified element as you would use a quick navigation key on a computer.
Unfortunately, there is not much auditory indication that a selected element is not available on a page when one is using the Rotor control. If “heading navigation” is selected and there are no headings, you may hear a thunk when you try to use the up or down flick. Some users may prefer a message such as “no headings” to indicate that their input has been received.
It is possible to set up the iPhone to check a personal or work email account. The Post Office Protocol (POP3), IMAP, and Microsoft Exchange protocols are supported. Email accounts can either be set up and transferred through iTunes or be configured in the Settings application under Mail, Contacts, and Calendars. Default server profiles exist for several common email providers, or the user can manually enter the information for a provider not listed. There are some problems with the email account set-up in the 3.1 version of the iPhone OS. Username fields are not correctly labeled when you are setting up an IMAP account, and it has been reported that no fields are labeled when your are configuring a Microsoft Exchange account.
Once a mail account has been configured, it can be accessed through the Mail application. This is typically found on the home screen as the second-from-the-left icon on the bottom row of applications. In addition to the title “Mail,” the icon displays and speaks the number of unread messages.
When opened, the email application places the user in the most recently accessed folder. It is possible to use the left and right flick gestures to move through the list of messages or slide a finger up and down to scroll with the scroll gestures. When a message is double-tapped, it is displayed on its own page. Icons are present at the bottom of the screen to refresh, move, delete, reply to the message, and compose a new message. If you wish to forward a message, activate the reply icon, and you will find options for Reply, Reply All, and Forward.
In an email message the Rotor control can be used to move through elements, including HTML, if the message is written in that format.
If you delete a message while it is displayed, the next message in your inbox will be displayed. Occasionally VoiceOver is not properly refreshed with the new content if you use the flick left and right gestures. You have to tap your finger at the beginning of the new message’s body and proceed from there.
From the inbox view it is possible to delete multiple messages. Activate the Edit button and double-tap any messages you would like to delete. The messages are selected, and the Delete button at the bottom of the screen updates to reflect the number of messages to be deleted.
One of the iPhone’s key selling points is its multimedia features. The iPod application is where many of the media tasks are accomplished. As with many other iPhone applications, a set of icons at the bottom of the screen provides a quick way to access common functions. By default these icons are Playlists, Artists, Songs, Videos, and More. Tapping one of the first four buttons will bring up the media matching that parameter. Tapping the More icon will present a list of additional media categories including Albums, Audio Books, Compilations, Composers, Genres, and Podcasts. All of the categories make use of the metadata provided by multimedia files to sort and display content. The media content and the associated metadata are transferred to the iPhone when it is connected to a computer running iTunes as described earlier in this article.
When a media track begins playing, you can press the Home button and return to the Home screen to perform other tasks. When VoiceOver is speaking, the media file’s sound will duck under the VoiceOver sound so you can hear the speech clearly. If you receive a phone call while the iPod application is playing a file, the file will pause until the call is complete. At any time you can double-tap with two fingers to pause or resume playback.
While the iPhone offers many advanced and powerful features of a handheld computer, it is also a telephone, and I would be remiss if I didn’t discuss that function. The Phone app can be accessed from the static bar of apps at the bottom of any home screen. Alternatively, the Home button can be pressed twice quickly, which will also launch the application. There are five components to the phone: Favorites (the iPhone equivalent of speed dial, where you can designate up to fifty numbers for quick access), Recents (displaying recent calls), Contacts, Keypad (allowing manual entry of a phone number), and Voicemail. The Recents and Voicemail icons include a number of unseen events if they are present, and that number will be announced following the name of the icon.
Favorites, Recents, and Contacts are all list views and can be navigated accordingly. The contacts screen also provides a search bar at the top to locate a contact quickly. Once a contact is selected, double-tapping any programmed number dials it. Double-tapping a number in Favorites or Recents automatically dials that number as well.
The Keypad allows one to dial a number that is not stored in memory. In addition to the standard telephone numeric keypad, buttons for Call and Delete are present below the zero and pound keys. The Delete key deletes the number to the left of the insertion point.
While you are on a call, icons on the screen allow you to perform actions including muting the call, placing the caller on hold, conferencing in another caller, and bringing up the keypad to interact with touch-tone-controlled systems. If the phone is pulled away from the user’s ear, the speaker phone automatically activates. It does not seem to be possible to use the on-screen icons while the speaker phone is off. Using the headphone that is included provided the best experience for me when dealing with an automated customer service or voicemail system because the VoiceOver speech is loud enough to be heard, and this is not always the case when using the speaker phone.
The iPhone has a unique way of accessing Voicemail. The voicemail system is called Visual Voicemail. When you activate the voicemail section of the Phone app, you see a list of all voicemail messages. The list indicates whether the message is unheard, the caller’s identity, and the time the message was left. Double-tapping the message causes it to be played through the phone’s earpiece. If you wish to play the message through the speaker phone, first activate the Speaker button located before the message list. A delete button is located near the bottom to the right of the application. This deletes the currently selected voicemail message.
When an incoming call is received, the caller ID is spoken. Depending on the volume of the selected ringtone, you may wish to silence the ring by pressing the Power button once and tap or flick to repeat the caller ID. Double-tapping the answer button connects the call. You can also use a two-finger double-tap to answer and end calls.
The iPhone offers developers a chance to create applications that take advantage of the power and high-speed data connection offered on the iPhone platform. Applications can be downloaded to the phone through iTunes or directly by using the App Store application on the phone itself. As with desktop applications, the method of development determines how accessible the application is to users of VoiceOver. Apple provides guidelines and tools to allow developers to build their applications so that they are accessible, but the developer must make use of these tools.
Commonly, inaccessible applications simply present unlabeled icons as “button.” In some cases it is possible to figure out the use of these buttons, but in others the lack of identification renders the application completely inaccessible. Depending on the application, the unlabeled buttons may be graphics that are not meant for interaction. For example, the Twittelator Pro application (a client for the popular Twitter social network) has unlabeled buttons placed between user posts, timestamps, and usernames, but these buttons do not appear to perform functions that are unavailable elsewhere.
In some applications users are flicking through a list when the cursor gets stuck and repeats the same item over and over. This can be remedied by tapping a finger on the next list item, but it is annoying when it occurs. This behavior is seen commonly in the Newsfeed section of the Facebook app.
Many applications, however, are completely accessible and present no challenges to the blind user. For example, the Around Me app, which uses your GPS position to locate nearby places of interest, presented no barriers to access. You can select from categories, including gas stations, hotels, restaurants, or all nearby places, and receive a list of matches. For historical monuments and landmarks, a brief description from the Internet is also included. Other accessible applications include the Sirius/XM Player, and the Pandora radio application.
This article has by no means been a look at all the applications and features of the iPhone 3GS, but I hope it has provided an idea of the user experience a blind VoiceOver user encounters. While a few challenges and barriers still exist in some applications, Apple has managed to provide nonvisual access to a mainstream device at no extra cost to the blind user. If you are considering purchasing an iPhone, I encourage you to visit an Apple or AT&T store and spend a few minutes with your hands on the phone. You will be able to get a feel for the navigation and user experience. Many resources, including the very active viphone user group, a mailing list for blind iPhone users, which can be found at <groups.google.com>, are available for users to ask questions and learn more about the product as well. Apple’s efforts in this area are quite positive, and we look forward to other manufacturers following in their footsteps to provide access solutions as part of mainstream products.