Team members (l-r): Maxwell Kofman, Leena Mathur, Surya Nehra and Nisha Chatwani. Photo/Aaron John Balana.
About 6 million people in the US are currently living with Alzheimer’s disease, the most common form of dementia, according to the Alzheimer’s Association. Despite being the sixth-leading cause of death in the country, there is currently no known cure for the memory-robbing condition.
But diagnosing the disease early can help people seek preventative care and slow its progress. That’s why a team of students at USC is developing machine learning tools to detect early-onset Alzheimer’s disease using speech patterns, and democratize the diagnosis process.
The team started working on the system in spring 2021 as a project for CAIS++, the student branch of the Center for Artificial Intelligence in Society, in collaboration with students from MEDesign, the biomedical engineering design group.
Led by computer science undergraduate students Leena Mathur and Nisha Chatwani, the team conducted machine learning research to analyze how a person’s speech patterns and word choices could help automated systems diagnose Alzheimer’s disease.
“We were inspired to start this project because we found the problem of dementia diagnosis compelling, specifically the development of low-cost, non-invasive, and scalable systems that can do this effectively,” said Mathur.
Diagnosing Alzheimer’s
To diagnose Alzheimer’s, doctors conduct tests to assess memory impairment and other thinking skills. One such test is known as the Cookie Theft picture test. In this test, participants are shown a picture of a woman washing dishes as the sink overflows, and two children steal cookies from a jar.
“It’s a very standard cognitive test for dementia diagnosis,” said Chatwani. “When people in healthcare settings are trying to diagnose people for Alzheimer’s, they’ll show people pictures like this and will look for clues in their speech patterns with respect to describing a story.”
The cookie theft picture from the Boston Diagnostic Aphasia Examination, commonly used in Alzheimer’s diagnosis.
Patients are asked to describe what they see, while medical professionals analyze their speech patterns to diagnose whether the patient has Alzheimer’s. But this diagnosis process can be costly and take several months to complete. What’s more, approximately 58% of the 44 million people afflicted with Alzheimer’s worldwide reside in less developed countries where diagnosis and treatment are not readily available.
Looking for red flags
In contrast, the student team’s automated dementia detection system offers a low-cost diagnosis method using machine learning. To train their machine learning models, the students obtained a dataset from a National Institute of Health study conducted at the University of Pittsburgh, comprising 293 participants. Audio clips and transcripts of participants describing the stimulus image were put through the machine learning model created by the student team.
The AI model analyzed patterns in these clips, looking for indicators of Alzheimer’s dementia. The model used information from two different modalities, or communication channels, in the audio clips: The words and psycholinguistic cues used by the patient (the “verbal modality”) and the acoustic aspects of a patient’s voice when speaking (the “audio modality”).
According to Mathur, the model was able to predict the future onset of Alzheimer’s in 76% of cases with the verbal modality, and 74% of cases with the audio modality.
“For example, our feature extraction captures aspects of verbal structure that psychologists have linked to analytic thinking, such as the structure and use of prepositions and conjunctions,” said Mathur, a computer science, cognitive science and linguistics major who focuses on research in human-centered artificial intelligence.
“People with Alzheimer’s dementia, while responding to the stimulus photo, leveraged language that was significantly less indicative of analytic thinking. In addition, participants with Alzheimer’s tended to use the past tense significantly more than the control group, which informed our models.”
An interdisciplinary team
In total, 11 undergraduates and one master’s student from 8 majors across the USC Viterbi School of Engineering and the Dornsife College of Letters, Arts and Sciences contributed to the project. Fittingly, the team won the Interdisciplinary Prize at USC’s Spring 2021 Undergraduate Research Symposium.
“Collaboration with so many people has been somewhat challenging, but it’s been a great experience,” said Mathur. “After we graduate, all of us will be working on huge teams either in research or industry, so being able to work on a project with 12 people was a valuable opportunity”
The Clinical Alzheimer’s Dementia Diagnosis project will continue to evolve this semester. So far, the team has focused on models that use information from audio and verbal modalities separately. Moving forward, they will be exploring multimodal approaches that fuse and align information from both modalities for more effective diagnoses.
“USC is one of the leading places in the world researching Alzheimer’s and other types of dementia, and there’s a rich field of multimodal machine learning that can inform Alzheimer’s diagnosis systems,” said Mathur. “We are excited to work on implementing our novel approaches this semester.”
Published on November 12th, 2021
Last updated on November 12th, 2021
