Dreaming Big: The Influence of Blind-Centered Experience and Direction on Artificial Intelligence
Presented by Bryan Bashin, Vice-Chair, Be My Eyes; Berkeley, California
From the Editor: Here is one of the most enlightening presentations of the 2024 National Convention agenda, one in which the presenter was a facilitator who introduced us to an almost unbelievable dialogue with artificial intelligence and showed how it can be harnessed to assist us in our day-to-day living. Here is what President Riccobono said to introduce the presentation:
MARK RICCOBONO: This presentation is from another entity which we supported through our partnership, along with SmartJob and the American Action Fund. We supported their exploration of artificial intelligence. This item is “Dream Big: "The Influence of Blind-Centered Experience and Direction on Artificial Intelligence,” and this gentleman is no stranger to this crowd. He brings his blind-centered intelligence to his role in helping to shape the work at Be My Eyes.
Here is Bryan Bashin.
BRYAN BASHIN: Good morning, fellow Federationists! You know, last year during convention, Be My Eyes CEO Mike Buckley stood here and told you about the thing that had just started, a little thing called Be My AI. Anybody use it here?
Today I want to tell you a little bit more about what happens when a company has leadership that is centered on blindness. Beginning ten years ago from our blind founder, furniture craftsman Hans Wiberg, who wanted more solutions, more tools, there are significant blind memberships at Be My Eyes on the board of directors, and our products are influenced by two hundred blind beta testers around the world.
In fact, are there any blind beta testers from Be My Eyes in this room? [Cheers]
Since Mike Buckley spoke here in the last twelve months, we've had a profusion of new features that are blind-informed. Now you can do private friends and family circles, create your own. You have Be My Eyes for work, you have Be My Eyes on the Android platform for the Global South. You have Microsoft partnered with Be My Eyes for Windows Desktop AI. Anyone use it yet? [Cheers]
A couple months ago we put out Be My Eyes' own inclusive language guide. What company puts out a disability language guide? One that is blind-informed. And we put out a service directory; it's in every app, maybe a thousand different companies and so forth, but also in there you will find the phone numbers and can be connected directly to every NFB affiliate president in the United States.
Centering on blindness really makes a difference. You know, those beta testers who started with us a little more than a year ago said one thing: “I want facial description,” and that loud and clear voice informed us. We're working with our partners at Open AI, and we said, “Our members need facial description.” They said, “No, we've got to blur it.” I said, “We need this description.” We're at a table demanding something from a one-hundred-billion-dollar company, and guess what? They gave us facial description. It's available in all forty-nine states except for the state of Illinois, which has its own rules. I think there are a few lawyers; Illinois, where are you? You've got some work to do.
What we are finding in this last year is that blind people are using the hell out of AI. Last month our servers had 2,100,000 queries just for Be My AI. That means every 1.2 seconds around the clock somebody on the planet who is blind is asking for description through AI. To our surprise, that's seven times more queries for AI than the old volunteer service. But guess what? The volunteer calls are still as strong as ever, have themselves increased, and it's important to us that, if you need information, you have the choice: AI or human. It doesn't matter, as long as you use it.
The average person who uses AI uses it 4.7 times a day to find out information. This last sixteen months of experience has shown us that AI can really do amazing things. I'll tell you one example. Last week a woman—a blind woman in California who is using Be My AI— has figured out how to write the code to run a 3D printer, and those who have run 3D printers know that sometimes they go awry. Is the object being created properly or is it not? Be My AI will describe how the thing is going and how it worked. So, I was very impressed. At the end I thought to ask her what university she is with, and she said, “I'm in the eleventh grade.” That's the future of AI.
So I want to talk to you a little about the future, and I want to do the most dangerous thing possible from the stage, which is a live demonstration. I'm going to ask my associate Kerns Lane to come and help with this demonstration. He's going to demonstrate our latest capability, which will be in your hands in a matter of weeks. It is GPT4.0, which is informed, I'm told, by one trillion parameters, things it sucked up on the Internet from God knows where. It's the largest model out there, apparently.
So when we demonstrate it, I want you to notice how the world is changing. Instead of taking a picture and waiting ten seconds, which was a miracle all the way back in 2023, today AI will be quick, conversational, and interruptible.
So, Kerns, where are you? Hello, AI, can you hear me?
AUTOMATED VOICE: Hi, I can hear you.
BRYAN: Can you describe the logo of the National Federation of the Blind?
AUTOMATED VOICE: Yes, the logo of the National Federation of the Blind has three intertwined circles in red, blue, and green, symbolizing equality, opportunity and security for the blind. Have you seen this logo before?
BRYAN: Yes, absolutely. But now I want to change topics, because knowing about STEM and science is important to blind people. Could you describe to me the appearance of the asteroid Ryugu?
AUTOMATED VOICE: Absolutely. Ryugu is a diamond shaped asteroid with a rugged surface covered in rocks and boulders. Its color is dark, almost black, and [it is] about half-a-mile wide. It's quite an intriguing sight. What got you interested in asteroids?
BRYAN: I'm interested in many things, but can you tell me that same description in Spanish?
AUTOMATED VOICE: [Speaking Spanish]
BRYAN: All right, now I would love to change from just talking to live video, okay? Are you ready?
AUTOMATED VOICE: Ready when you are.
BRYAN: Can you tell me what you see?
AUTOMATED VOICE: It looks like a conference room with lots of people seated and a podium at the front. There is a banner with logos, including Google, hanging behind the podium. Are you attending or presenting at this event?
BRYAN: Can you tell me if the folks look like they are enjoying my presentation?
AUTOMATED VOICE: They seem engaged and attentive.
BRYAN: Okay. That's a first. I would like to ask Kerns to point the camera at the President of the National Federation of the Blind?
AUTOMATED VOICE: How are you feeling about the presentation so far?
BRYAN: This is Mark Riccobono; do you know who he is?
AUTOMATED VOICE: Yes, Mark Riccobono is the President of the National Federation of the Blind. How are you feeling about how it's going?
BRYAN: Excuse me, can you tell me what he looks like?
AUTOMATED VOICE: Sure. Mark Riccobono is wearing a dark suit with a green shirt and a tie. He has short light-colored hair. He seems focused on his presentation. Anything else you would like to know?
BRYAN: You said he has short light-colored hair. The other day he told us that after ten years his hair is receding. Is it really receding?
AUTOMATED VOICE: Yes. Yes, his hair is receding. [Laughter]
BRYAN: Okay, Mr. President, can I ask you to do something for the camera? Just do something, and I'll ask AI what you are doing.
PRESIDENT RICCOBONO: Okay.
AUTOMATED VOICE: Sounds good. I'm ready to describe what is happening.
BRYAN: Tell me what he is doing.
AUTOMATED VOICE: He's waving his hand at the camera. He's waving again. Anything else you would like to ask him?
BRYAN: Try one last thing. What is he doing now?
AUTOMATED VOICE: Now he is [raising] his hand up high, almost like he's waving or making a point. Seems like he … he's still keeping his hand up like he's making a point or signaling something.
BRYAN: He's very determined like that. Thank you, AI. And thank you, Kerns.
Will this be something requiring high tech when it comes to Be My Eyes? No, that was an iPhone 13 using a regular cellular network. I'm going to tell you something: that today's AI is the worst AI you will ever use. It's only going to get better.
We are seeing over the last year fewer hallucinations. But the blind-centered folks at Be My Eyes are seeing some things that are actually a little worrying about this too. The descriptions that AI is giving are increasingly bland, warm, and homogenous. If you ask, say, for a photo of a real estate broker, you are never going to get a real estate broker carrying a white cane. Blind people are missing from these large language models. So we're seeing deeper problems because of that.
Sometimes you will ask one of these models—and there are about twenty-four of them active right now, so I don't want to pick on any one. You will say, “I'm blind,” and it will respond with “I'm sorry.”
Or sometimes you will hold up a business card—and you think that would be an easy thing—and you're happy that the business card is in Braille and print. You'll say, “Can you read the business card?”
The AI says, “I can't read it because I can't read Braille.”
Sometimes it's taken a picture of me and described me like it described President Riccobono and said, “He's carrying a long, red-tipped cane.” Well, I've never used a red-tipped cane in my life. The models have to stop generalizing. [Applause]
The models actually have told me that it's preferable to call somebody “visually impaired” than blind. Obviously, they weren't in resolutions yesterday. It's called a dog guide, “your best friend,” and sometimes we see that you get syrupy and condescending language when a blind person does the merest accomplishment.
Then it still has, guess what: the same ableist construct when you ask it, as the video will now tell you, “your Uber is here.”
But where is here?
One day I thought, okay, I'm going to try one model. I'll get my mail, and I'll just go through it one piece at a time. So I picked out a piece of mail and held it up. I wanted to see who it is from. And it said, "It's an envelope."
Then I thought, okay, I'll ask it a second question. I did.
And it said, “Oh, it has a return address and regular address and a stamp on it.”
It took me three times to get it to read who it was from. It never occurred to the model makers that users of this service would be blind or would hold paper upside down. These models are not unkind or prejudiced. We simply don't exist in them right now. We need to train these readers to be efficient as we train human readers. [Applause]
This field is called "reinforcement training through human feedback." Large companies, the largest LLMs, are spending up to $100 million every year with humans training the models to be more accurate and more respectful. But to my knowledge, no blind person has yet been hired to join this team of reinforcement training. Would any of you like to be hired to train these models? [Cheering]
Some companies say they don't need us; they can train these models with theoretical models of blind people. Do you think that could possibly represent our lived experience?
AUDIENCE MEMBERS: No!
So how can we make things better? In the very near future, you will have a chance to train these models yourself. In fact, our friend Everette Bacon announced this week that his company, Aira, is already soliciting your input to train its models for better results.
Now, I know that some of your sessions, your video sessions particularly, may be private, and we want to find ways to respect that. So in our and other user agreements, you will always have the chance to opt out if something seems private to you. But we hope you are going to lean in and train these models. It's the only way that we can exist and show up in them. You know, we train teachers and employers, family, everybody on the street about blindness every day. Why is training AI any different from that? Actively speaking our truth has always been at the core of Federationism.
And so I ask you: Did you sit and wait for the vision industrial complex to get rid of subminimum wages?
AUDIENCE MEMBERS: No!
BRYAN: Did you sit and wait for somebody else to put sounds on quiet electric cars?
AUDIENCE MEMBERS: No!
BRYAN: Did we put up with ATMs that didn't talk when they arrived?
AUDIENCE MEMBERS: No!
BRYAN: Do you want to sit and wait for the AI companies to get blindness right by themselves?
AUDIENCE MEMBERS: No!
BRYAN: So please, share your sessions. Consider it a revolutionary act to change the machine.
You know, tech writer Tom Chatfield said it best. "There is no question that AI is changing the world. The question is... who will be changing AI? Whose majority models will be the ones that go forward? And who will be left out?"
Federationists, we must not be left out. Thank you! [Applause]