A computer graphics and virtual reality expert, Hao Li has an extensive track record of developing new capture and immersive technologies for virtual training and simulation. Photo/Noe Montes.
USC computer science assistant professor Hao Li, a computer graphics and virtual reality expert, has received an Office of Naval Research (ONR) Young Investigator Award to create highly realistic computer-generated (CG) humans for immersive training purposes.
Li is among 31 scientists selected from more than 340 applicants to receive this highly competitive award, which supports outstanding early-career scientists and academics. The three-year grant totaling almost $600,000 will support Li’s project, “Complete Human Digitization and Unconstrained Performance Capture.”
Specifically, Li will use deep-learning-based 3D digitization and synthesis methods to rapidly and cost-effectively create high-fidelity CG humans with virtual clothing and props.
Despite major advances in computer-generated imagery (CGI), producing believable CG characters with clothing and accessories is a time-consuming and expensive task that often relies on complex 3D scanning equipment and a lengthy design process led by skilled digital artists.
“While computational models exist for anatomical structures such as human bodies and faces, they cannot be extended to handle the immense range of garment types, appearances, and their deformations,” said Li, holder of the Andrew and Erna Viterbi Early Career Chair. As such, current human digitization techniques are limited to unclothed subjects.
Li’s proposed research enables the creation of complete clothed avatars for virtual training and simulation applications at scale, relevant to next-generation Naval science and technology capabilities. Photo/Hao Li, ONR YIP technical proposal.
Virtual Worlds
To tackle this challenge, Li aims to firstly reconstruct a complete human avatar model from a 3D input scan of a person with separate geometric models for clothing and accessories.
Then comes the AI: deep-learning frameworks will be developed to “learn” the extensive shape variations of garments and props, after being trained on similar data sets including 3D acquisitions of real Marines with different types of uniforms and equipment.
Li’s proposed research, which uses consumer-grade depth sensors and cameras, could be used to scale digital asset creations for immersive virtual training and simulation in the U.S. Navy. The 3D capture technique could even potentially displace traditional video conferencing, with conference participants represented by realistic 3D virtual avatars.
“The ONR Young Investigator Program award is an important recognition of our research in photorealistic human digitization using recent advancements in deep learning,” said Li.
“I’m honored to be one of the recipients. Our efforts will enable new possibilities for creating highly compelling digital content at scale for next generation AR and VR-based experiences. More fundamentally, this research grant will spark new foundations in the field of AI-driven computer graphics.”
Li has an extensive track record of developing new capture and immersive technologies for virtual training and simulation. His research has set the foundations for the modern treatment of 3D human digitization, facial performance capture, data-driven modeling, 3D deep learning, and virtual reality.
Born in Germany, he earned his doctorate from ETH Zurich and his master’s from the University of Karlsruhe (TH). Prior to joining USC, he was a lead research scientist at Lucasfilm’s Industrial Light & Magic, and a postdoctoral researcher at Columbia and Princeton Universities. In addition to his tenure-track faculty position, Li also leads the Vision & Graphics Lab (VGL) at the USC Institute for Creative Technologies (ICT).
In 2013, he was named a top 35 innovator under 35 by MIT Technology Review and in 2015 he won the Google Faculty Research Award and the Okawa Foundation Research Grant. He is also the co-founder and CEO of Pinscreen, an augmented reality startup focusing on photo-realistic avatar technology.
Published on March 1st, 2018
Last updated on April 8th, 2021
