Apps created by Embody Digital, Ari Shapiro’s startup, allow users to create lifelike avatars that look and speak like them. (Image/Courtesy of Ari Shapiro)
Think of your day-to-day activities. You are probably doing a million and one things, such as going to school, working, picking up your dog from the groomer, making dinner, and responding to a seemingly infinite number of emails or text messages.
Ari Shapiro – USC Viterbi research assistant professor of computer science head of the Character Animation and Simulation research group at the USC Institute for Creative Technologies (ICT) – has come up with a solution that allows harried, hurried people to be in many places at all hours – at least in the digital world.
Shapiro and his team have created smartphone apps, UBeBot for Android and AR Avatar Director for iPhones, that allow users to customize their own realistic looking avatars, which can emulate their voice, gestures, and physical traits.
The need for these two apps stems from this “evolution of communication,” which is appealing to “people who are crafting personalized video content for social applications like Tik Tok, Twitter, Instagram, or similar,” Shapiro said.
This evolution began with the emoji, which provides users with one-dimensional character expressions like happiness, sadness, embarrassment, or confusion. Then bitmoji was born, which allows users to create their own cartoon avatar. However, Shapiro’s apps not only allow its users to customize their own digital character, but also make it talk with the user’s tone of voice and move its body and make gestures.
“Our research shows that the embodiment makes a difference. These embodied characters are better liked, better remembered, better understood, and can be more convincing than just disembodied voices or text or emojis,” Shapiro said.
These apps – created by Embody Digital, Shapiro’s startup with technology licensed from USC – are unique for their ability to engage the user in the creation process of the avatar, not only visually, but also auditorily. Here’s how they work.
UBeBot, which was created almost seven months ago, allows the user to take a photo of themselves and create a 3-D avatar with it. Then the user records their voice so the avatar can perfectly imitate their tone. Additionally, the user can type a phrase or sentence, which the avatar can “speak” with movements or gestures of the user’s choosing.
By contrast, 18-month-old AR Avatar Director for iOS, lets the consumer choose from pre-designed cartoon-like avatars and customize hair and eye color. The user can also record their voice and choose different gestures, so the character speaks and moves like the user.
Working with Shapiro on the avatar apps are Anton Leuski, an expert in conversational AI and a research scientist for USC ICT, and Stacy Marsella, a specialist in body language and computational behavior modeling, who now teaches at the University of Glasgow.
Shapiro believes that communication through human-like, interactive digital entities will take the place of existing communication through text, voice, and replace some direct human interaction. As businesses seek greater efficiency, they will replace human with bots and AI, yielding bot-powered order-takers at fast food restaurants, AI-generated bank teller agents, and in-store assistants to help shoppers locate specific items.
“The ability to replicate a person digitally and make them available at any time for multiple people at once will become a disruptive force in health care, entertainment and personal interaction,” Shapiro said.
An example of the potential impact of avatars on healthcare is an app built by the USC Center for Body Computing and Shapiro’s team at USC ICT. The app allows a patient recently diagnosed with cardiac arrhythmia to access, 24/7, a digital version of their doctor in that doctor’s likeness with that doctor’s voice answers questions about the diagnosis, whether or not it is safe to work out, and treatment options, among others. The patient establishes a rapport with their own doctor through the digital construct, and gets access to basic information from the digital doctor whenever they want. Additionally, the real doctor can practice better medicine because they no longer have to spend the majority of their time answering questions during patient visits.
For entertainment, celebrities could employ personalized avatars to represent them to the world. Fans could finally “talk” with these untouchable celebrities beyond social media. From the celebrities pre-programmed responses to simple questions like his or her upbringing or first performance, the user can “receive a customized answer, generated through the AI,” Shapiro said.
Similarly, avatars could be customized to represent deceased relatives and their responses, appearance, with beloved family member programming their behavior. “What if you could get advice, ask about their childhood, or elicit their thoughts long after they have passed?” Shapiro said.
Published on August 22nd, 2019
Last updated on May 17th, 2022
