VoCreate

Empowering artists & designers with hand impairments

Timeline

9 months
Sep 2022 - May 2023

Team

Michael Tranquilli - SME
Eric Forman - Advisor
Liz Danzico - Chair

Role

Voice UX
Co-Designing Workshops
Prototyping and Testing

Overview

VoCreate is an assistive drawing app that lets artists and designers draw using their voices.

VoCreate offers a voice-user interface specifically for art creation, revolutionizing how we approach artistic expression. VoCreate empowers individuals with motor impairments by providing a novel way to bring their ideas and creativity to life through drawing.

The Problem

How might we empower artists and designers with motor impairments to draw with more accuracy and independence?

Motor Impairments Limit Hand Control Significantly

Injuries like Spinal Cord Injury impact the person's spine which can result in life-long paralysis of arm, hands, and legs. This impacts a person's ability to use common objects, including pen/stylus, mouse, or keyboard.

User Goal: Empower artists to draw with more control

Limiting Current Methods Result in Frustration

While splints (image above) allow users to hold a pencil or a stylus, it does not allow them to control it accurately as users are mostly able to control their shoulders. Also, it takes longer to get used to.

User Goal: An easier to learning and use method

Introducing VoCreate

An assistive drawing app that allows artists to draw with their voice

VoCreate is  designed meticulously with the input of artists with hand impairments, to ensure that it meets their needs and allows them to express themselves freely.

Voice-First Interface

Users can control the app using their voice, such as vowel sounds to move the brush and paint. By making the "EE" sound the user can move the paint brush to the left and by making the "AW" sound to the right. It's fun, but also useful when drawing longer strokes while maintaining artistic control.

Accessible Navigation

The redesigned tool menu allows users to access drawing tools from anywhere on the screen, making it more accessible than a fixed toolbar. This reduces the need to constantly reach for frequently used tools, saving users energy and breath.

Understanding Artist's Recovery Journey

Artists impacted by injuries that limit their drawing abilities resolved by using voice

VoCreate was co-designed with artists who have experienced spinal cord injuries resulting in paralysis from the shoulders down. While it typically takes 6 months to 3 years for these artists to return to their creativity, VoCreate enables them to start drawing within weeks by using their voice, the most quickly recovered ability.

User Journey Map identifying where in user's journey can the experience be improved

Defining the Voice User Interface

Testing Different Methods of controlling paintbrushes using voice

The goal was to test different voice UX methods and identify the ones which resolved the user's problems more accurately. Based on 40 rigorous (and fun) user tests, vowel-joystick proved to be the most accurate.

Directional Method

Using directions to move the paintbrush was the obvious method. However, this method was limiting in movement, non-continuous, and required user to speak in words.

Vowel-Sounds

Using vowel sounds to move the cursor was slightly difficult to learn, but after 20 minutes of practice users were able to use it more fluently. It was more accurate, organic, and required less effort to use.

Organic Paintbrush Motion

As different vowel sounds mix well together, users were able to draw organic shapes, unachievable by any other method in voice interactions.

Scaling Sounds Functions

Beyond vowel-sounds, users could interact with the operating system using other sounds to perform common functions.

User Testing the Drawing Experiene

Redefining the Canvas Experience

Once the interaction was confirmed, the drawing canvas experience needed to be designed. Using card-sorting and affinity mapping I identified and sorted 40 features that needed to be in the system.

Standard UI was Incompatible with Voice UX

The first user interface was based on standard design patterns in the arts software industry with a modern look and feel. However, the interaction was not highly compatible with voice interaction.

Redefined UX Patterns for Voice

I redesigned each component to take be more compatible with Voice UX. I worked on improving navigation accessibility, feedback loops, and simplifying the UI resulting in a more usable and desirable experience.

Impact of a Co-Designed Solution

VoCreate was revolutionary compared to current products

Users provided highly positive feedback and recommended VoCreate over currently available common solutions such as hand splints and eye-tracking, claiming it was quicker, easier, and more independent.

Comparison of key user needs between VoCreate and most common current solutions.

“I can imagine using this product during the recovery... this would've brought much joy to my then isolated life.”

Simon, Musician, Spinal Cord Injury for 2 years

“I've been injured for 10 years and no one ever involved me in designing products for my own needs. I really enjoyed the process.”

Jack, Artist, Spinal Cord Injury for 10 years

Key Learnings

Including users from the start

Co-designing is a valuable method to eliminate biases and assumptions by designing with users, not for them.

Designing Voice-UX System

Designing a Voice-UX system, I established new standards in an emerging field, mastering sounds, speech, volume, and tone.

Accepting the unconventional

By focusing on user-centered research, unconventional solutions can emerge, revolutionizing established standards.

Conclusion

VoCreate is a testament to the power of user-centered design and technology's ability to empower marginalized communities. By understanding the unique needs and challenges faced by artists and designers with hand impairments, we were able to create a tool that not only empowers them to draw unassisted but also brings joy back into their creative lives. This case study illustrates the impact of accessible design and the potential of voice-first interactions in transforming the lives of individuals facing significant challenges.