VoCreate

Voice UI
Accessibility

A voice-controlled drawing application for artists with motor impairments using AI to translate human sounds to computer interactions

Problem

Artists with motor impairments are unable to use physical tools accurately and comfortably for an extended period of time.

Impact

75%

Reduced setup time by 75% by removing the need to setup assistive technologies resulting in a more independent user experience.

85%

Received positive feedback from users, with 85% reporting an improved experience and higher satisfaction.

View Full Case Study

Role

UX Research and Design

Deliverables

Interactive Prototype
User Research
Co-Design Workshops
Research Documentation
Product Mock-up and Presentation

Duration

Aug 22 - Mar 23 (9 months)

Tools

Figma
Miro
Voice Flow
Adobe After Effects
Adobe InDesign
Adobe Illustrator

Overview

VoCreate is an assistive drawing app that lets artists and designers draw using their voices.

VoCreate offers a voice-user interface specifically for art creation, revolutionizing how we approach artistic expression. VoCreate empowers individuals with motor impairments by providing a novel way to bring their ideas and creativity to life through drawing.

Problem Statement

How might we empower artists and designers with motor impairments to draw with more accuracy and independence?

Motor Impairments Limit Hand Control Significantly

Injuries like Spinal Cord Injury impact the person's spine which can result in life-long paralysis of arm, hands, and legs. This impacts a person's ability to use common objects, including pen/stylus, mouse, or keyboard.

User Goal: Empower artists to draw with more control

Limiting Current Methods Result in Frustration

While splints (image above) allow users to hold a pencil or a stylus, it does not allow them to control it accurately as users are mostly able to control their shoulders. Also, it takes longer to get used to.

User Goal: An easier to learning and use method

Solution

VoCreate is a voice-controlled drawing experience that allows artists with hand motor impairments to draw

VoCreate is  designed meticulously with the input of artists with hand impairments, to ensure that it meets their needs and allows them to express themselves freely.

Voice-First Interface

Users can control the app using their voice, such as vowel sounds to move the brush and paint. By making the "EE" sound the user can move the paint brush to the left and by making the "AW" sound to the right. It's fun, but also useful when drawing longer strokes while maintaining artistic control.

Accessible Navigation

The redesigned tool menu allows users to access drawing tools from anywhere on the screen, making it more accessible than a fixed toolbar. This reduces the need to constantly reach for frequently used tools, saving users energy and breath.

User Research

User journey revealed that artists found it difficult getting back to drawing after a traumatic injury.

VoCreate was co-designed with artists who have experienced spinal cord injuries resulting in paralysis from the shoulders down. While it typically takes 6 months to 3 years for these artists to return to their creativity, VoCreate enables them to start drawing within weeks by using their voice, the most quickly recovered ability.

ideation & Concepts

Testing Different Methods of controlling paintbrushes using voice

The goal was to test different voice UX methods and identify the ones which resolved the user's problems more accurately. Based on 40 rigorous (and fun) user tests, vowel-joystick proved to be the most accurate.

Directional Method

Using directions to move the paintbrush was the obvious method. However, this method was limiting in movement, non-continuous, and required user to speak in words.

Vowel-Sounds

Using vowel sounds to move the cursor was slightly difficult to learn, but after 20 minutes of practice users were able to use it more fluently. It was more accurate, organic, and required less effort to use.

Organic Paintbrush Motion

As different vowel sounds mix well together, users were able to draw organic shapes, unachievable by any other method in voice interactions.

Scaling Sounds Functions

Beyond vowel-sounds, users could interact with the operating system using other sounds to perform common functions.

Design Iterations

Redefining the Canvas Experience

Once the interaction was confirmed, the drawing canvas experience needed to be designed. Using card-sorting and affinity mapping I identified and sorted 40 features that needed to be in the system.

Standard UI was Incompatible with Voice UX

The first user interface was based on standard design patterns in the arts software industry with a modern look and feel. However, the interaction was not highly compatible with voice interaction.

Redefined UX Patterns for Voice

I redesigned each component to take be more compatible with Voice UX. I worked on improving navigation accessibility, feedback loops, and simplifying the UI resulting in a more usable and desirable experience.

Impact

We user tested the concept with multiple artists with spinal cord injuries which revealed that VoCreate was 20x more usable.

Users provided highly positive feedback and recommended VoCreate over currently available common solutions such as hand splints and eye-tracking, claiming it was quicker, easier, and more independent.

Comparison of key user needs between VoCreate and most common current solutions.

“I can imagine using this product during the recovery... this would've brought much joy to my then isolated life.”

Simon, Musician, Spinal Cord Injury for 2 years

“I've been injured for 10 years and no one ever involved me in designing products for my own needs. I really enjoyed the process.”

Jack, Artist, Spinal Cord Injury for 10 years

Key Learnings

Including users from the start

Co-designing is a valuable method to eliminate biases and assumptions by designing with users, not for them.

Designing Voice-UX System

Designing a Voice-UX system, I established new standards in an emerging field, mastering sounds, speech, volume, and tone.

Accepting the unconventional

By focusing on user-centered research, unconventional solutions can emerge, revolutionizing established standards.

Conclusion

VoCreate is a testament to the power of user-centered design and technology's ability to empower marginalized communities. By understanding the unique needs and challenges faced by artists and designers with hand impairments, we were able to create a tool that not only empowers them to draw unassisted but also brings joy back into their creative lives. This case study illustrates the impact of accessible design and the potential of voice-first interactions in transforming the lives of individuals facing significant challenges.

Let's Create Something Magical

LinkedInEmail

2024 Syed Hussain. All rights reserved.