David Nelson, director of ICT’s Mixed Reality Research Lab
Picture this scenario.
Sergeant Goodwin, a seasoned sexual assault response coordinator (SARC) for the military, sits across from a young soldier named Specialist Smith.
Sgt. Goodwin is known for her empathetic and compassionate approach in dealing with victims of sexual assault. She’s conducted numerous interviews over the years, each one unique and challenging in its own way.
This time, however, the situation leaves her deeply unsettled. Specialist Smith is laughing uncontrollably as she describes the moment she became a victim of sexual assault by her own command leader.
Sgt. Goodwin is puzzled. She has seen victims react with nervous laughter or inappropriate humor as a defense mechanism before, but this seems different.
How is Sgt. Goodwin to respond? What’s the most appropriate thing for her to say? How could she come across as supportive and professional at the same time?
The USC Institute for Creative Technologies is providing the military with answers to tough questions like these through its Digital Interactive Victim Intake Simulation (DIVIS) project.
You can think of DIVIS like you would a flight simulator for pilots where SARCs and victim advocates (VAs) in the military are able to train and practice their skills in a realistic way through virtual interviews done with a digital victim.
“The goal of DIVIS has been to create a standardized, interactive simulation learning experiences that enable SHARP (Sexual Harrasment/Assault Response and Prevention) responders to practice interpersonal skills like rapport building and active listening,” said David Nelson, director of ICT’s Mixed Reality Research Lab and DIVIS project lead.
“We collect data on the trainees while they’re interviewing the digital victim to see if they’re mentioning the things they’re supposed to mention or if they’re behaving the way they’re supposed to. DIVIS also provides actionable feedback on their performance to help SARCs increase their confidence in responding to difficult situations in real life.”
According to the U.S. Department of Defense, in 2018 an estimated 20,500 service members experienced some form of sexual assault, representing about 13,000 women and 7,500 men.
Nearly one in four servicewomen report experiencing sexual assault in the military, according to a meta-analysis of 69 studies published in 2018 in the journal Trauma, Violence & Abuse.
The D.O.D. has been executing new initiatives and training methods like DIVIS to help lower sexual assault cases in the military. By 2022 it received 8,942 reports of sexual assault involving service members as victims and/or subjects.
Since 2019, USC ICT has been working with the Army’s SHARP Academy to develop DIVIS – leveraging several research groups across ICT to bring the simulation system to life.
“DIVIS is a truly collaborative effort at ICT,” Nelson explained. “We integrate the capabilities developed by ICT researchers in natural language, user-perception, learning sciences and mixed-reality to create a highly responsive, interpersonal social simulation.”
The goal of the simulation system is to recreate lifelike, immersive situations for those in the SARC and victim advocacy field, so that victims feel that their stories are heard and appropriately handled.
Within the training setting, trainees are positioned at a desk facing a large widescreen television. To enhance the authenticity of emotionally charged scenarios and make it seem as realistic as possible, Nelson’s team enlisted the services of professional actors to be recorded and depicted as digital victims.
These digital victims materialize on the television screen, adept at conveying subtle intricacies of emotional interactions, such as non-verbal cues and micro-expressions. Thanks to natural language technology used in DIVIS, the trainee is then able to speak with the digital victim as if they are a real human – asking them questions and getting natural responses.
“The natural language interface enables the trainee to interact with the character in a simple conversational way,” said Nelson. “One of the ways we keep the system realistic and adaptive is we film the actors in a variety of emotional states, so that a trainee can work with the same digital victim multiple times and have very different results.”
While DIVIS is not an official military program of record yet, SHARP trainers are piloting the system, using it as part of their six-week training program at Fort Leavenworth in Kansas.
“We have received very positive feedback from both trainees and instructors,” said Nelson.
“In fact, the SHARP Academy hired ICT to produce two additional victim characters and scenarios to incorporate into the DIVIS application.”
“Trainees frequently say that their experiences with the DIVIS system are some of the most challenging intake interviews they’ve ever done. They sweat, the hair on the back of their necks stand up, they feel like they are interacting with a real victim of sexual assault.”
“Trainees frequently say that their experiences with the DIVIS system are some of the most challenging intake interviews they’ve ever done. They sweat, the hair on the back of their necks stand up, they feel like they are interacting with a real victim of sexual assault.”
David Nelson, director of ICT’s Mixed Reality Research Lab
Another important component of DIVIS is the after-action review which allows an instructor to review the entire simulation with the trainee. ICT researchers designed a user interface that creates a synchronized video recording of both the trainee and the digital victim during the interview.
The instructor can then scrub through the timeline to review relevant moments within the interaction and discuss opportunities for improvement and learning. The system also uses natural language recognition to identify keywords that the trainee used during the conversation.
ICT’s Omnisense multi-modal sensing capabilities analyze facial expressions, gaze and body posture to gauge things like whether the trainee was actively listening and maintaining eye contact or inappropriately smiling.
David Cobbins, Alesia Gainer, and David Traum served as co-leaders on the DIVIS project.
Nelson and Cobbins also share screenwriting credentials. Collaboratively, they crafted the scenarios and scripted all of the digital victim’s dialogues, drawing upon their narrative expertise to connect the realms of entertainment and technology in order to address tangible issues. In fact, prior to joining ICT, Nelson accrued two decades of experience as an award-winning producer and director, contributing to various projects involving rapper Tupac Shakur and filmmaker Spike Lee.
“ICT is a very special place,” said Nelson. “The mandate in the founding of ICT 25 years ago was to bridge storytellers from the entertainment industry, computer scientists and game designers – putting them all together to solve problems for the D.O.D. and the world at large.”
ICT is now working with the SHARP Academy to make DIVIS a program of record for the military, enabling the simulation learning tool to be deployed at different Army bases around the country and around the world, while providing better help for victims of sexual assault in the military.
Published on November 8th, 2023
Last updated on November 8th, 2023
