Illustration: A blind woman in a kitchen making food and speaking to a smart home device with speaker bubbles above the two of them respectively

How to Design Voice-Controlled Smart Devices for Blind People

Reading time: estimated 7 minutes

On This Page

Thomas Logan, Equal Entry, Found and CEO. He documents some of the most common accessibility barriers for blind people in the context of using voice-controlled smart devices. Then he further speaks to the creators of voice-control products, voice assistants, and voice applications about accessibility considerations.

Introduction

Many of the popular IoT devices for consumers are meant to create a smart home. That’s because almost any electronic device or appliance in a home can act as a smart home device. Already, we can buy everything from programmable lights and thermostats to doorbells to dishwashers and much more.

According to multiple resources in the IoT Wikipedia entry, IoT is defined as “physical objects that are embedded with sensors, processing abilities, software, and other technologies [that] connect and exchange data with other devices and systems over the Internet or other communications networks.”

However, IoT can create accessibility barriers by: 

  • Containing touch screens with no alternative input
  • Using physical buttons without tactile labels 
  • Displaying information visually with no other options
  • Having no sound output 
  • Failing to allow for speech input 

Many of these challenges can be overcome for blind and low vision customers as well as for people with other disabilities and even no disabilities by adding support for voice control in these smart devices.

Voice-Enabled Smart Device Benefits

A voice interface can open the door for people who are blind by making it possible for them to better control their technology. In comparison to how screen readers work on the web, it turns out that the speech output of information can be too verbose and time consuming to process. 

Because a screen reader customer must use the keyboard to navigate a web page first to find the action or intent they want to take, they end up spending time listening to more information than needed. Also, the inherent nature of audio being a single channel of information that is presented linearly, it is difficult to efficiently navigate large streams of text that are common on today’s websites. 

With voice control, the customer can say what they want to do with their device. And then, they’ll receive speech output confirming that the action is done or another voice prompt asking for more information. 

In the same way that an interface can be made accessible to voice by implementing an accessibility application programming interface (API), smart devices can be enabled for voice through a combination of IoT devices on a network. So, a smart device won’t need its own speaker or microphone to use this functionality.  

Customers can control their IoT by using voice assistants such as Siri, Alexa, Cortana, and Google. These voice assistants are available on Windows, MacOS, iOS, and Android platforms. The advantage of these is that people won’t need to purchase another device to use voice functionality. 

Consumers can also purchase popular standalone products such as Google Home, Amazon Echo, and Apple HomePod. These products provide more flexibility for issuing voice commands to perform actions from anywhere in the home. For example, customers may want one of these devices in their kitchen to help with cooking.

Google Home, Amazon Echo, and Apple HomePod devices displayed side by side

Device Considerations 

How do we ensure any IoT device can work well for voice? Someone evaluating the hardware will review its functionality such as buttons, dials, and switches that give the device its capabilities. All of its functions should be turned into voice commands that are easy for people to use.

Here are examples of functions that can be voice-controlled:

  • Turning a product on or off such as turning on the lights in the living room.  
  • Checking whether a product is on such as asking if the oven is on or off.
  • Finding out what the visual status indicators show such as asking the laundry machine how much time is left before the wash completes.  
  • Changing a value such as requesting a thermostat to set the temperature to 72 degrees. 
  • Selecting from options such as telling the dryer to set the cycle to permanent press. 

If you want to get started, consider experimenting with or using a tool to build your voice app. One platform that helps teams design, prototype, and launch voice and chats apps is Voiceflow. Using this or a similar tool you can review whether you have fully accounted for all of the functionality your hardware contains and ensure that it can be accessed via voice.

Accessibility Considerations

Before you move ahead to development, keep in mind that it’s good practice to design your voice interface and include accessibility considerations before development begins. It will be harder and more expensive to make changes later in the process.

Here is a checklist of questions to ask about whether a connected device is accessible:

  1. Have you confirmed that every function you can do on the physical device can also be done with voice input? 
  2. Have you tested that a variety of common voice phrases will work to control your device? 
  3. Have you had people with disabilities test your voice interface to confirm it can support a variety of styles of voice commands to complete the action? 
  4. Have you tested how long it takes a user to complete a task using their voice? Can you optimize the required voice input and output to make the user more efficient?
  5. Have you tested how many steps it takes to complete a task on your device? Is there an opportunity to cut down the steps required to complete a task? 
  6. Have you published accessible documentation with all the commands that can be used to control your device?  Check out this example of  Google’s instructions on how to Cook with Google Assistant.  
  7. Have you provided accessible documentation on how a user can install or turn on the technology that allows your device to be controlled by voice?

Real-World Example: Cooking with Voice Control 

Let’s take a real world example of how a blind person can benefit from using a voice assistant.  Let’s cook! 

We need to use our hands while we are preparing meals. Using a Voice assistant is an excellent tool to focus our attention on cooking rather than reading instructions. 

In my kitchen, I had the following exchange with Google Echo:

  • [Thomas] Hey Google, find me a guacamole recipe.
  • [Google Home] OK, I’ve got a recipe called Guacamole from AllRecipes, this recipe serves four and takes about 10 minutes to make. Does that sound good?
  • [Thomas] Yes.
  • [Google Home] Great, would you like to start with ingredients or instructions? 
  • [Thomas] Ingredients.
  • [Google Home] There are eight ingredients. You can ask for the next ingredient, or skip to the instructions.  The first ingredient is three avocados, peeled, pitted, and mashed. 
  • [Thomas] What’s the next ingredient?
  • [Google Home] <No Response> 
  • [Thomas] Hey Google, what’s the next ingredient?
  • [Google Home] The second ingredient is one lime, juiced.
  • [Thomas] Hey Google, how much of the lime?
  • [Google Home] Sorry I don’t understand.
  • [Thomas] Hey Google, what is step number 1?
  • [Google Home] The first ingredient is three avocados, peeled, pitted, and mashed.
  • [Thomas] OK Google, what is the next step?
  • [Google Home] The second ingredient is one lime, juiced.

Some notes from my experience creating this simple workflow. It is easier when the Google Assistant is able to continue listening to me and stay in the intent that I am preparing something to cook. I noticed that at times I had to redirect the Google assistant to pay particular attention to my command because it stopped listening.  

For example, when I said “What’s the next ingredient?” the Google Home stayed silent. It felt natural to keep talking to my assistant because I was in the process of cooking. I had to take additional time to say, “Hey Google, what’s the next ingredient” and at that point the assistant did remember that I was in the process of making guacamole. This is definitely one area that can be improved.

The other part relates to the authoring of recipes that work well for voice assistants. The same way that we have the Authoring Tool Accessibility Guidelines for the web, we should have these for developing content that works well for voice assistants. Not every recipe has a nice, unordered list of ingredients and an ordered list of steps to create the recipe. This should be documented and recommended to people who create recipes that they need to build their recipes that would work effectively  for a voice assistant.

As an example, look at a pasta recipe. Most pasta recipes contain instructions to cook the pasta and instructions to cook the sauce. For a recipe optimized for a voice assistant, it is best to create a single set of steps that need to be completed sequentially.

This means that maybe you start making your sauce at the same time you’re making pasta. Assuming that someone is looking visually at your recipe will be a design no-no. By making each step sequential, you are helping anyone who is listening to your recipe to successfully do it in the shortest amount of time. 

This means it would be more efficient for recipe authors to present the steps for their recipe exactly as they perform them in the kitchen. They should not structure them in a way that may assume someone is visually reviewing the recipes.

The Next Steps for Voice-Controlled Devices

It takes a new set of skills to author voice interfaces that are easy to follow. The process for listening to and responding to a voice prompt is different from reading text on a device or pressing a physical button. Planning and testing with people with disabilities are essential to ensuring that voice-controlled experiences are efficient and comfortable.  

All IoT device manufacturers need to implement APIs such as Apple Home Kit, Amazon Alexa Skills, and Google Assistant Actions to make it possible for consumers to control their products with their voice and listening. Be aware that this may require separate developer tasks to implement the best voice interface for each platform. As this field matures, it will grow easier to build a voice interface once and have it work across a plethora of devices and platforms. 

Hardware and IoT designers who want to be thought leaders and gain a competitive advantage are designing interfaces that work well for people who are blind. It would be especially beneficial if they take the time to write about their process of how they created successful experiences and who they worked with to make it happen. 

The voice control capabilities of the Internet of things (IoT) can empower people who are blind or low vision. This accessibility tool will give them a way to control the devices in their lives.  The key is that the devices must be thoughtfully designed with accessibility in mind from the beginning.

Note: If you would like to dive deeper into designing voice interfaces, check out Preston So’s “Voice Content and Usability” book.