Joshi is one 21 graduate students from around the world to be recognized by this year’s fellowship. Photo/Alexis Situ.
Brihi Joshi, a third-year computer science student, has been named a 2024 Apple Scholar in artificial intelligence and machine learning (AI/ML). She is one of just 21 graduate students from around the world to be recognized by this year’s fellowship, which aims to support the academic research community “by amplifying emerging leaders in their field and their cutting-edge machine learning research.”
Joshi works with the Natural Language Processing (NLP) Group under the direction of Xiang Ren, the Andrew and Erna Viterbi Early Career Chair and an associate professor of computer science, and Swabha Swayamdipta, a WiSE Gabilan assistant professor of computer science.
Joshi’s research is focused on the symbiotic relationship between humans and large language models (LLMs) — how can humans effectively use LLMs and in turn, how can LLMs benefit from human intervention? To do so, Joshi often uses model explanations — a description to explain how an AI system arrived at a particular interpretation — to help users understand the meaning behind a model’s outputs.
Her goal is to investigate whether LLMs, such as the model that powers ChatGPT, are useful to people in diverse settings, from laypeople in everyday life to domain experts, such as doctors or lawyers, in their professions.
We sat down with Joshi to find out more about her, her work, and the fellowship.
Tell me about yourself!
I’m originally from New Delhi, India. I completed my undergrad in India, where I worked a lot in social media analysis, for instance, understanding patterns from mass human communication on social networks. Now, since LLMs started coming into play, a lot of research in our group at USC has gone into understanding how they work.
Why did you decide to pursue a PhD?
I worked as a software engineer for a while, and while that is important and relevant — a lot of things that people use in their daily lives are based on engineering — I just didn’t feel the excitement. I felt like we were constrained in what we could think and invent. And I think that’s the benefit of PhD programming in academia: You get to broaden your horizons, thinking about things people haven’t thought about before.
What’s your research about?
LLMs are everywhere these days, but are they actually useful for humans? Everyone says ChatGPT is great, but what is it great for? Who is using ChatGPT? How are they using it and is it actually helping them? There’s a big research gap and I’m interested in understanding the utility of these LLMs in a more principled manner. Can we quantify that utility? Can we improve that utility?
Let’s ask ChatGPT a science question: Why is the sky blue? It might give you a five-page, physics-based answer to this. If a layperson is looking at the answer, it won’t make any sense. Right now, I’m researching whether ChatGPT can understand who is interacting with it. Is it somebody with a PhD in physics? Is it a layperson who studied physics 10 years ago and doesn’t remember some of the concepts? Is it a 10-year-old child? ChatGPT’s answer should change based on who’s interacting with the model, and that will determine whether the answer is useful.
How do you quantify whether something is useful to someone?
Like any research question, there is a broad idea of what quantifies utility, but you must scope it down. In one of my papers, I measure utility by looking at questions that are relatively tough for humans to answer.
I show a couple of human annotators this question: “Is Australia bigger than the United States?” They can’t answer that question, so they guess.
I show them an explanation is generated by an LLM, like “Australia is 7.7 million square kilometers, and the United States is 9.8 million square kilometers.” If the explanation is useful, humans will be able to answer the question correctly after seeing the explanation. If the explanation is not useful, the next question is, “How do we make it useful?”
Interesting. Tell us more about the Apple Scholar in AIML PhD Fellowship!
This is a way for Apple to support PhD students in their research, recognizing their work and giving them mentorship opportunities. Apart from generous funding for a PhD, they provide us with a mentor to help us brainstorm — be a sounding board in industry — as well as internship opportunities and networking connections.
What are you looking forward to most as an Apple Scholar?
I absolutely love the idea of being connected to a mentor. I have great PhD advisors and they are great mentors, but the Apple fellowship will provide an industry perspective. The LLM community is very fast-paced right now. Every day, there are a bajillion new research papers and problems that people are working on, which is great, but then that gives you very little time and opportunity to carve a niche for yourself. Having that industry mentor will help me ground my work in something that people actually use.
Are you looking to go into industry after you finish your PhD?
I don’t have an answer for that … *laughs* … I’m interested in industry and research, so every avenue is open to me right now.
Finally, a bit of a random question: If you weren’t a computer scientist, what would you be?
Hands down, I would be a musician! I would probably be a singer-songwriter. I’ve taken a lot of music classes at USC, and I’m part of the USC Concert Choir. I have taken multiple piano and drum classes here, and currently taking pop performance. Next semester, I’m taking songwriting. It just keeps me sane!
Published on April 23rd, 2024
Last updated on April 23rd, 2024
