The Vitisense Project
Vitisense is a seamless mobile authentication app using intuitive biometrics (e.g. habitual gestures like picking up the phone).
This was my design project for my Masters degree in product design.
The project took 3 big iterations over the years from 2012 to 2014, each with its own research-design-feedback sprint. The design process involved user research that explored the goals and characteristics of a potential Vitisense user, and the concepts and values of this solution from the perspective of the user. Of course, there was academic research into the technologies and algorithms to make this system feasible. I also developed prototypes using all of this information and tested the system to validate this solution of seamless mobile security.
The following sections describe the user experience, design and business ideas that brought this project to life.
Death to typing Passwords!
When I worked at Blackberry, we loved to eat our own dog food! 🐶 Legend has it that employees have extra set of eyes on their heads because their real eyes was locked onto these “crackberries”. We probably checked our smartphone more than 150 times a day!
The company had an emphasis on security and privacy, so every phone required PIN access. In total, I would spend about 5 minutes a day punching PIN codes on a small screen or keyboard. What was worse is the context switch disrupting my productivity (as described by the American Psychological Association).
To top it off, the security of text-based passwords is a contentious subject. Consider these statistics:
► The majority of people, 64%, have written down their passwords
► 82% of people have forgotten their passwords. Additionally, there are numerous human error “cracks” where password can be reset without stringent user identification.
► Approximately 9% of 4-digit PINs can be cracked in three tries
Amidst almost getting locked out of my business phone again, I had an eureka moment.
What if the phone unlocked just by the way I lifted it?
The Vitisense project was born.
The Before & After Picture
This is a story of smartphone users with and without biometric authentication:
Make it POP
To kick off this project, I did some rapid prototyping by drawing sketches and tested the interactions using Prototype on Paper mockup mobile app. This prototype was fantastic for retrieving quick feedback from fellow designers at UXWaterloo Meetup and colleagues.
Back It Up
At first glance, the frustration of typing passwords may seem like a very personal problem, which is not enough to confirm product-market fit or feasibility. Given that this was an academic project, it was appropriate that I conduct some research into the current gesture recognition concepts, biometrics collection technologies, machine learning algorithms, and the social acceptability of biometric technologies.
Few interesting tidbits I discovered through my research:
Our bodies gives off all kinds of unique biometrics, from iris patterns, the rhythm of our typing, to our gait. These biometrics can allow for many ways to identify a human being.
There are two general types of biometrics - physiological and behavioural. Physiological biometrics can be fingerprints, iris patterns, etc., and are usually unchangeable. Behavioural biometrics such as keystroke and gait can change over time. There are different risks with each type. For example, if a fingerprint is stolen, the security is breached permanently and cannot use the same print again. On the other hand, a dynamic biometric may be less secure because the matching process may be less accurate.
Gesture recognition is feasible through camera and other sensor technologies. There are numerous papers dating back to the 1980s confirming these applications. In spite of this, machine learning for gesture recognition in real time is still very bleeding edge. So, seamless authentication may be a challenge.
Gesture recognition with movement sensors such as gyroscope and accelerometers is relatively new, but processing these signals uses the same conventional image recognition and compression methods.
Biometric authentication is a very different subject from biometric identification. Authentication involves additional measures to consider security and privacy risks. Unfortunately, there had been little published research on biometric authentication. Fortunately, it is a trending topic so there should be more insight within the next 5-10 years.
People do not like to make grand gestures that make them look like a clown. There are gestures that are socially acceptable and gestures that are offensive to different cultures. Moreover, there are subtle gestures that may come across as inappropriate in context (e.g. Check your watch too early in a meeting will annoy the presenter, tapping your foot, etc.)
Feed It Back
Armed with my research and basic prototypes, I ventured out for initial feedback from UXWaterloo community group and the McMaster Innovation Showcase.
Designers from UXWaterloo thought it was a cool solution to a relevant problem they have experienced firsthand. They were an excellent resource in exploring ideas and design challenges, such as managing asynchronous signals, concept feasibility, using different sensors, preventing forgeries, etc. They definitely gave me a lot of concepts to mull over late nights.
McMaster’s Innovation showcase’s aim that year aimed to highlight a project’s commercial prospect, social impact, and groundbreaking research. One of the judges evaluated Vitisense to have good amount of the first two attributes. 👍 The feedback I received was more business-oriented. Judges and spectators commented on protecting the intellectual property, data and trademarks.
There was skepticism around the feasibility of the project given the state of consumer technology in 2012, and there was some hesitance with technology monitoring you all the time (like a wearable).
The conversation that piqued my interest was that Vitisense is a great idea for accessible security, people with low dexterity or visual impairment can afford security through their regular habits and movements.
At this point, the Vitisense ecosystem has started to take shape. I was able to converge on key ideas and build on them with a purpose. Very exciting!
Personify
All this great feedback allowed me to form ideas about who are the users and what are their goals in the Vitisense ecosystem. So, I created some personas of potential users of Vitisense and their reason for incorporating Vitisense into their digital lives.
Stakeholders of Project Vitisense
Leaps & Bounds
Now that I had an idea of what Vitisense was and could be, I could set the goals and scope of the project.
The Big Idea
Effortless mobile security — Privacy and security should not obstruct a user’s focus to meaningful content.
Goals
► Evaluate value of this technology to customers
► To eliminate difficult to use and time consuming passwords – seamless unobtrusive security (i.e. Eliminate conscious password input)
► Implement biometric authentication through arm gestures
► Research the uniqueness of arm gesture for identification
► Research the level of security for gesture authentication – accuracy (internal performance), in comparison to other consumer market products, in working environments
► Make technology usable on mobile phone
User Testing: Achievement Unlocked!
Like all good scientists, I tested my research and confirmed that Vitisense is feasible!
This was the most challenging (and scary 😰) part of the project because the outcome was unpredictable. Dynamic biometric authentication is cutting-edge technology. My personal obstacles were that I need to churn the research into testable targets, learn to implement AI libraries, and develop Android apps to test with potential users.
At this point, the project was beyond the scope of the graduate requirements, but I was committed to making this app into reality because I want to complete a full design cycle experience and see the value of my work.
Test Objectives
► Gather user data to analyze each individual’s unique signature
► Gather user data on forgery attempts
► Gather user feedback about their feelings on the concept, the product, and the training process
Ethical Human Trials Enables High Quality Data
Having the project cleared by the university’s ethics committee was actually a great learning experience in the planning for user testing. Designers should not only consider the effects of their products on the individuals, but also the effects in the environment the product is being used. Setting a safe and controlled space for user testing also standardized the experience for clear quantitative data to use in analysis.
Vitisense Alpha v. 0.24
As a part of the user testing, I built a rough Android app that gave the participants instructions, and collected the participants biometric data. The reasoning to have the app give the instructions was to minimize my influence and bias as a facilitator. Moreover, I wrote the instructions in a casual and fun tone that could be used in the actual app calibration training. This way, I can also observe the participants’ reactions to the possible feel of the product.
Test It Out
The user trials were one-on-one sessions set up in a casual but quiet space that was familiar and comfortable to the user (i.e. living rooms, dining rooms, study, etc.) I had 5 participants and each session took approximately 20 minutes to complete.
The mobile app was set with 3 types of activities for the user to follow: recording their personal signature swing, performing a gesture from a written description, and mimicking a gesture that I give them. These gesture recordings are used to confirm uniqueness of personal signature, and the difficulty level in forgery.
From the results, I was able to confirm that people do pick up the phone in their unique way. There was scepticism around the limit on number of ways a user can pick up a device, and that would lead to “naive” or accidental forgeries. However, I observed that every participant retrieved their phone differently: some picked it up face up vs. face down, some picked up the phone from the desk vs. the purse vs. the pant pocket, some picks up the phone with one hand or switches hands mid-way, and some unlocks the phone portrait vs. landscape mode. All these variables are registered as unique identifiers by the accelerometer and gyroscope sensors in the device.
Even when the participant was trained to forge a gesture, the biometric data still came out sufficiently unique because everyone’s arm length, the swing speeds, and infliction points were different. The forgery exercises further validated that arm gestures are unique to each individual. We are all special! 😊
How did it feel?
After the gesture recordings, I surveyed the participants and discussed how they felt learning and performing the gestures, and how they felt about this product concept. When there were unexpected or interesting discussion points, I would go off-the-tangent from the survey and ask the participant to elaborate. I found this method of conversation (i.e. standardized survey plus exploratory discussion) was highly engaging and very valuable because I can better empathize with their experience. As well, I received feedback that help confirm expectations, and feedback on unpredicted but important issues.
Ironically, I discovered that participants felt “robotic” when they tried to train the system to learn their gesture, which counters the idea of natural feel for the product. The repetitive motions also felt tedious. This is very enlightening feedback that tells me the Vitisense system needs a better methods to engage with the user and learn the user’s gestures. Therefore, I think I need to revisit and prioritize the idea of training on-the-go with machine learning to get authentic gestures.
All participants agreed that they did not want any barriers to accessing their devices. The Vitisense concept did provide a new method of securing their device, and they do believe it is easier than other current methods. However, convenience trumped security. So much so that a couple of participants do not lock their phone because they want quick access.
One participant’s reason to leaving his phone unlocked was that he surrendered to the open data trend. He felt that his personal data was easily retrievable everywhere and web companies tracked his every move online, and these intruders could not be stopped. Furthermore, he was willing to give up privacy when the service provided was valuable. I found his reasoning has become the “wicked” problem for consumer behaviour and security design today.
Information Architecture
I created this workflow to fully visualize the connections between user actions to user's tasks, and to the back-end processes and services. This is the overview of Vitisense app's capabilities.
Mock it up
Ta-da!
All that research and creativity is boiled-down into this mock-up. This interactive mock-up will walk through how Vitisense appears as a running process, and the experience of Vitisense settings app.
The main colours are turquoise greens, to represent living biometric data, and slightly desaturated primary colours to tie-in the familiarity of DNA representation in science diagrams. To maintain the Android KitKat look-and-feel, the type, the font colours, and the settings layout is the default Android styles.
Go on, play!
Feedback - Round #3
So this is what a dissertation may feel like. 😅
I presented my finished prototype and research findings to three engineering professors at McMaster University. The consensus was that my design was extremely comprehensive and well-received. Woot!
The professors appreciated that I applied many concepts of design thinking, from building personas to testing with live human subjects. They were also excited about the viability of this project as a business venture. They definitely understood the value of seamless security because they have handled sensitive information at work. Of course, there were more big ideas to address such as rights to privacy, building secure data storage, market fit and value, etc.
There was some apprehension from the guiding professor with regards to the prolonged length of time for project completion. The reason is I exceeded the initial scope of a design project in McMaster’s graduate program. I made a commitment to myself to build this project through to a proof of concept in order to present it as a feature item for my design portfolio. On the bright side, the project was also a valuable and constructive example for the professors to see the additional processes for a design project to come to full fruition, and thus, make curriculum improvements to this graduate program and bring more great ideas to life.
Onwards and Upwards
It is clear opportunities in biometric authentication has just began to surface. In order to grow Vitisense into a successful business, I would need to tackle some of these challenges:
⁌ Vitisense in the cloud — This can be implementing secure cloud storage for Vitisense’ collected biometric data, or building a new business idea of integrating biometric authentication with cloud services. For example, developing APIs for third parties to authenticate users with Vitisense data. No need to log into Facebook or email.
⁌ Integration with wearables — Expand Vitisense application from authenticating smartphone to authenticating other devices via wearables. No need to carry additional security passes to get into your work building.
⁌ B2B integrations — Similar to above ideas, Vitisense can be applied and partnered with other business services. Many companies value keeping intellectual property safe, Vitisense can provide seamless access to specific users on specific devices at specific times.
⁌ eHealth diagnostics and health care management — There are also potential opportunities in biometric information management in health care industry. For example, the biometric data may help doctors diagnose muscular problems quickly and keep track of healing progress.
Our world continues to become more and more connected, and the convenience of identifying oneself forms the basis for most of our online lives. Imagine if each one of these interactions was a little quicker, easier, and safer. What could we do with that extra time and decreased mental load?