The papers presented by USC researchers provide new insights on several topics related to AI and machine learning, from 3D object recognition to fairness in AI.
USC researchers are presenting a plethora of discoveries at the 37th annual Conference on Neural Information Processing Systems (NeurIPS), one of the world’s top AI conferences, held Dec. 10-16 in New Orleans, LA.
Topics include the creation of robots capable of learning tasks after a single demonstration, the development of smarter and more agile robotic systems, and the exploration of robots acquiring tasks after a single demonstration in computer simulations.
Research spotlights
Interpreting the world in 3D
“3D Copy-Paste: Physical Plausible Indoor Object Insertion for Monocular 3D Object Detection”
Yunhao Ge, Hong-Xing Yu, Cheng Zhao, Yuliang Guo, Xinyu Huang, Liu Ren, Laurent Itti, Jiajun Wu
Copy and paste. It’s a simple concept. You define some text or image on your computer, copy it, and paste it where you want it. A team of USC computer science researchers has now developed a similar technique to copy virtual 3D objects and paste them into real indoor scenes. This creates an overall natural and realistic image in terms of spatial relationships, object orientations and lighting.
The technique – called 3D Copy-Paste – can teach computers how to recognize the virtual 3D object in a multitude of different settings without having to rely on the tedious and expensive process of having a human feed the computer with reams of data. In short, 3D Copy-Paste can improve how computers see and interpret things in 3D space. This technique could have “profound” implications for both the computer graphics and computer vision fields, said the researchers, including applications in autonomous driving technology.
Learning quickly in new environments
“RoboCLIP: One Demonstration is Enough to Learn Robot Policies”
Sumedh Anand Sontakke, Jesse Zhang, Séb Arnold, Karl Pertsch, Erdem Biyik, Dorsa Sadigh, Chelsea Finn, Laurent Itti
A new algorithm developed by USC computer science researchers shows that robots in computer simulations can learn tasks after a single demonstration, performing two to three times better than other imitation learning (IL) methods.
In this simulation from the adjacent paper, the robot successfully closes the drawer with a single video demonstration/text description.
Aging populations and caregivers stand to benefit the most from future work based on the algorithm, called RoboCLIP, which dramatically reduces how much data is needed to train robots by allowing anyone to interact with them through language or videos.
Future research is needed before this study translates into a world where robots can learn quickly with few demonstrations or instructions – such as fetching a tall glass of chilled water – but RoboCLIP represents a significant step forward in imitation learning (IL) research.
Right now, IL methods require many demonstrations, massive datasets, and substantial human supervision for a robot to master a task in computer simulations. Now it can learn from just one.
Smarter, more agile robots
“Generating Behaviorally Diverse Policies with Latent Diffusion Models”
Shashank Hegde, Sumeet Batra, K.R. Zentner, Gaurav S. Sukhatme
USC computer science researchers are working on simultaneously solving two problems in robotics: How to make deep neural networks (DNNs) efficient enough to control robots on a millisecond-to-millisecond basis, and how to control DNNs with natural language.
Doctoral student K.R. Zentner, who co-authored the study, explains the process:
Video from the adjacent paper shows differences in movement over time and robust self-balancing behavior, achieved by generating and running four different small neural networks sequentially.
“We trained many small neural networks to control a simple walking robot in a minimal–but very efficient–simulated environment, generating many neural networks that walk in different ways. Then, we described how each of these neural networks walked, for instance, ‘hop on left foot’ or ‘shuffle forward,’ producing a dataset of over a hundred different ways of walking.”
“Finally, we used this dataset to train one large neural network. This large neural network–a ‘hyper-network’–takes in a description of how to walk and produces a small neural network that walks in that way. We also showed that the large network can be 13x smaller than all of the small neural networks added together while keeping 98% of the performance and 89% of the variety.”“Because the hyper-network can be controlled using language, it opens a path to allowing robot behavior to be controlled by human operators or using much smarter—but slower—deep neural networks. Because the networks the hyper-network generates are small, they can be used to control a robot many times a second. Together, these properties might lead to robots that are smarter and more agile than those available today.”
All papers with USC affiliation
“Recovering Unbalanced Communities in the Stochastic Block Model with Application to Clustering with a Faulty Oracle”
Chandra Sekhar Mukherjee, Pan Peng, Jiapeng Zhang
Michael Hanna, Ollie Liu, Alexandre Variengien
“Practical Contextual Bandits with Feedback Graphs”
Mengxiao Zhang, Yuheng Zhang, Olga Vrousgou, Haipeng Luo, Paul Mineiro
“Regret Matching+: (In)Stability and Fast Convergence in Games”
Gabriele Farina, Julien Grand-Clément, Christian Kroer, Chung-Wei Lee, Haipeng Luo
“Uncoupled and Convergent Learning in Two-Player Zero-Sum Markov Games”
Yang Cai, Haipeng Luo, Chen-Yu Wei, Weiqiang Zheng
“FedGCN: Convergence-Communication Tradeoffs in Federated Training of Graph Convolutional Networks”
Yuhang Yao, Weizhao Jin, Srivatsan Ravi, Carlee Joe-Wong
“Hierarchical Gaussian Mixture based Task Generative Model for Robust Meta-Learning”
Yizhou Zhang, Jingchao Ni, Wei Cheng, Zhengzhang Chen, Liang Tong, Haifeng Chen, Yan Liu
“On the Power of SVD in Clustering Problems”
Xinyu Mao, Jiapeng Zhang
“No-Regret Online Reinforcement Learning with Adversarial Losses and Transitions”
Tiancheng Jin, Junyan Liu, Chloé Rouyer, William Chang, Chen-Yu Wei, Haipeng Luo
Tiancheng Jin, Junyan Liu, Haipeng Luo
“Asynchrony-Robust Collaborative Perception via Bird’s Eye View Flow”
Sizhe Wei, Yuxi Wei, Yue Hu, Yifan Lu, Yiqi Zhong, Siheng Chen, Ya Zhang
“SwiftSage: A Generative Agent with Fast and Slow Thinking for Complex Interactive Tasks”
Bill Yuchen Lin, Yicheng Fu, Karina Yang, Faeze Brahman, Shiyu Huang, Chandra Bhagavatula, Prithviraj Ammanabrolu, Yejin Choi, Xiang Ren
“TACO: Temporal Latent Action-Driven Contrastive Loss for Visual Reinforcement Learning”
Ruijie Zheng, Xiyao Wang, Yanchao Sun, Shuang Ma, Jieyu Zhao, Huazhe Xu, Hal Daumé III, Furong Huang
“H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models”
Zhenyu Zhang, Ying Sheng, Tianyi Zhou, Tianlong Chen, Lianmin Zheng, Ruisi Cai, Zhao Song, Yuandong Tian, Christopher Re, Clark Barrett, Zhangyang Wang, Beidi Chen
“Context-lumpable stochastic bandits”
Chung-Wei Lee, Qinghua Liu, Yasin Abbasi-Yadkori, Chi Jin, Tor Lattimore, Csaba Szepesvari
“LIMA: Less Is More for Alignment”
Chunting Zhou, Pengfei Liu, Puxin Xu, Srini Iyer, Jiao Sun, Yuning Mao, Xuezhe Ma, Avia Efrat, Ping Yu, LILI YU, Susan Zhang, Gargi Ghosh, Mike Lewis, Luke Zettlemoyer, Omer Levy
Sara Babakniya, Zalan Fabian, Chaoyang He, Mahdi Soltanolkotabi, Salman Avestimehr
“Learning via Look-Alike Clustering: A Precise Analysis of Model Generalization”
Adel Javanmard, Vahab Mirrokni
“Faith and Fate: Limits of Transformers on Compositionality”
Nouha Dziri, Ximing Lu, Melanie Sclar, Xiang Lorraine Li, Liwei Jiang, Bill Yuchen Lin, Sean Welleck, Peter West, Chandra Bhagavatula, Ronan Le Bras, Jena D. Hwang, Soumya Sanyal, Xiang Ren, Allyson Ettinger, Zaid Harchaoui, Yejin Choi
“Arbitrarily Scalable Environment Generators via Neural Cellular Automata”
Yulun Zhang, Matthew Christopher Fontaine, Varun Bhatt, Stefanos Nikolaidis, Jiaoyang Li
Arman Zharmagambetov, Brandon Amos, Aaron M Ferber, Taoan Huang, Bistra Dilkina, Yuandong Tian
“Optimal and Fair Encouragement Policy Evaluation and Learning”
Angela Zhou
“Learning Provably Robust Estimators for Inverse Problems via Jittering”
Anselm Krainovic, Mahdi Soltanolkotabi, Reinhard Heckel
“SOL: Sampling-based Optimal Linear bounding of arbitrary scalar functions”
Yuriy Biktairov, Jyotirmoy Deshmukh
“Derandomized novelty detection with FDR control via conformal e-values”
Meshi Bashari, Amir Epstein, Yaniv Romano, Matteo Sesia
Published on December 12th, 2023
Last updated on December 14th, 2023
