9 months
Sep 2022 - May 2023
Artists and designers with motor impairments affecting their arm, wrist, or hand who wish to create digitally must adapt to tools not designed for them. Many current solutions focus only on hardware, while software and user experience have been overlooked.
Although there are many talented artists and designers with motor impairments, current adaptive tools are not accessible to many who do not have precise control. Additionally, drawing software interfaces do not focus on the needs of this community.
As a result, even if people are interested in visual arts or design, professionally or as a hobbyist, they may get discouraged.
Introducing VoCreate, a drawing app for the iPad that empowers artists and designers with hand impairments to draw with their voice. The entire experience design meticulously with artists with hand impairments to be able to express themselves by empowering them.
VoCreate is designed to be used handsfree completely. Users can use different sounds to interact, such as vowel sounds to move and paint with the brush tool, and make other sounds like "ck" and "ch" for clicking and "pop" to open tools menu.
The tool menu was redesigned to help the user access different drawing tools from anywhere on the screen. This is more accessible than having a fixed location for the toolbar.
An important addition to the menu was to add the new canvas option to the top of the Gallery so it is accessible all the time. Also, using the similar design language on the menu to make inconsistent with the rest of the application.
I conducted 6 user interviews remotely with people with varied scale of motor impairments affecting their ability to control their hands, wrist, or arms. Users ranged from having temporary disability due to minor injuries to permanent disability due to spinal cord injury. Key insights were:
For the scope of the research, I focused on users with Spinal Cord Injury (SCI), I conducted literature review and desk research and identified problem space. Briefly, spinal cord is a nerve tissue column running down the back, protected by layers of tissue and surrounded by vertebrae. Injuries to the cervical region (neck) of the spinal cord can lead to impairment in controlling neck, shoulder, and arm movements. Such injuries cause quadriplegia, affecting everything below the injury level. Focus of the study was on injury levels from C4 - T1, which effected the persons ability to control their hands accurately.
To get myself trauma informed about SCI, I worked with a subject matter expert, Michael Tranquilli, an occupational therapist and accessibility professor at NYU with a decade of experience working with people with SCI.
Once the primary users were identified, I set a goal to involve people with lived experiences from the start so I am able to design with them. To achieve this, I designed Co-designing workshop sessions with 2 users with SCI for the duration of the research. The workshops were divided into four sessions:
To understand what adaptive tools artists with hand impairments are using, I analysed user experiences of 6 products and services. Splints were the most used adaptive tools, they are wooden or plastic appendage that allows artists to hold a pencil, stylus or brush. However, they are cumbersome to use as they require frequent assistance and do no assist with finer motor movements. Through this analysis I was able to identify key opportunities for innovation. Users wanted to draw more frequently, so they desired an experience that is easy to setup, assistance-free, allows finer motor control, and affordable.
One of they key outcomes of the research and discovery phase was the user journey map which was designed to empathise with the user's experience during the trauma recovery. At the initial stages, it is difficult to introduce any non-medical solutions as users need medical assistance. After the surgery, when the user is in their recovery phase, they feel lonely. Art therapy has proven beneficial in rehabilitation, but, the tools used in therapy are frustrating to use and learn.
Once the research hypothesis was derived, key assumptions were identified and categorized based on their importance. Assumptions that were critical to test were the important and unknown assumptions. The key assumption to be tested was the ability to draw with voice.
To test the hypothesis, we needed to test a primary assumption: are users comfortable drawing using their voice. To complete this test, I conducted 4 experiments with users, stakeholders, and experts. The goal was to determine how users wanted to draw using their voice. Key insights from these tests involved:
While our users are still used to contemporary art and design tools which are heavily reliant on complete control over what is created, it was interesting to see where such AI tools can help generate more ideas in less time at the cost of losing control. It was important to focus on what is the user's goal?
Drawing with AI transforms the entire concept of drawing from making to thinking. The basic interaction is having conversations with the system and based on those conversations the AI will generate images. Instead of voice mapped to interact with the computer, in this concept am questioning what it really means to think about creating something.
Through user testing and observation, it was revealed that having high control was desired. I tested two main methods of moving the paint brush on the screen:
After several user testing, Vowel Joystick was selected due to following advantages:
Once the interaction method was decided, sketches of the main screen was designed based on users mental model of a drawing tools. This was important to highlight the key features needed to provide a user-friendly experience. Every interaction was modified for voice only input method.
Based on the sketches, wireframes were designed using Figma. The goal was to test if the information and tools available meet user's expectations and metal model and is a viable solution to pursue.
The prototype was tested semi-moderated remote tests with 4 users. The goal was know whether the interaction was easy to understand and met user's expectations. As the entire system was supposed to run with voice, the users were only allowed to speak to interact with the wireframes and I performed the actions. They key results were:
Another key insight gained from the research was the user's trauma journey. To emphasise the user's live experience, illustrations were used to depict the stages of the user's journey instead of a map. As a result, stakeholders were able to empathise at a deeper level.
Based on the feedback on wireframes the first prototype was designed. The idea was to further test the concept in a more finished form. They key improvement were:
With the prototype, I conducted remote usability tests with 2 users during the Co-design workshops. Users were given tasks to operate the system with as little help as possible. Based on the user feedback, iterations were made. Key results of usability tests were as following:
The version 2 was the most important iteration reflecting the most critical feedbacks from the users. This included considering users’ reduced lung capacities due to the injury, their drawing environment with limited manoeuvrability, and immersive interface that allows user to have a delightful drawing experience.
Along with the interaction design, it was important identify a suitable visual language that will be consistent throughout the product. The key attributes affecting the decisions were: simple, fluid, and playful.
As a result of this research, the concept was able to achieve vastly better results than the splint, a standard tool used by artist and designers with hand impairments currently. In the comparison chart below, I compare the key results achieved by VoCreate that encourage users to draw more with less effort.
As the most important design project in my career so far, I have learned many valuable lessons throughout this journey. The first lesson is that I do not propose VoCreate as a solution that will solve the problem I started with but a concept that was built with some of the people in the community. Some other impactful lessons are:
During the research, I diverted most of my energy into understanding the behaviours, needs, and barriers of people with hand impairments. To do this, I had to research extensively to understand the very complexities of the problem space and make trauma informed decisions. I achieved this was by defining hypotheses and design methods to test key assumptions. This helped me streamline my research goals and allow a benchmark to measure research success.
As I designed for accessibility, I questioned my role as a designer in the whole process of user-centered designer. I encountered the most critical stance by Liza Jackson who warns about designing "innovative" solutions that end up being disability dongles the community is not interested in. While it will take me many more years to improve on this, for the thesis, I designed with my users as closely as possible given the resources to make sure their voices and perspective led the key design and experience decisions. As a result, I was able to foster genuine connection with the people and community and prototype interesting concepts which otherwise would not have been possible.
I believe voice interactions, or Conversation Design (CxD), is one of the most natural way of interacting with machines. With the advances in Artificial Intelligence and Large Language Models, there is a possibility we might be closer to interacting with machines similar to how we interact with each other, through conversations. Through this project, I was able to conceptualize one aspect of a person's language, vowel sounds, can perform one action, drawing, and how it can help marginalized communities to prosper. Future impact of this could mean artists with hand impairments could gain back their creative career and be more financially independent after the trauma. Additionally, similar interaction could enable such users to perform other complex tasks like driving their wheelchair or interacting with computers. The idea is to work with people in such communities to design better solutions with them, in whatever capacity they wish to use it.
.jpg)