Ruihan Yang

I'm currently a first second third-year PhD candidate at UC San Diego, advised by Prof Xiaolong Wang. Before coming to UC San Diego, I received my B.E. in Software Engineering from Nankai University in 2019

I'm interested in reinforcement learning, machine learning, robotics and some system stuff. Specifically, I'd like to build intelligent agent, which make decision with information coming from different sources

Email  /  CV  /  Google Scholar  /  Github  /  Linkedin

I'm open to discussion or collaboration. Feel free to drop me an email if you're interested in my research.

Going to visit Cambridge, MA in Fall 2024, Let me know if you would like to chat

profile photo
Publication & Preprint
* denotes equal contribution
Bunny-VisionPro: Real-Time Bimanual Dexterous Teleoperation for Imitation Learning
Runyu Ding*,  Yuzhe Qin*,  Jiyue Zhu*,  Chengzhe Jia,  Shiqi Yang, Ruihan Yang,  Xiaojuan Qi,  Xiaolong Wang

arXiv, 2024
arXiv / Project Page / Code

Bunny-VisionPro is a real-time bimanual teleoperation system that prioritizes safety and minimal delay. It also features human haptic feedback to enhance immersion.

SpatialRGPT: Grounded Spatial Reasoning in Vision-Language Models
An-Chieh Cheng,   Hongxu Yin,  Yang Fu,  Qiushan Guo, Ruihan Yang,   Jan Kautz,  Xiaolong Wang,  Sifei Liu

arXiv, 2024
arXiv / Project Page

A powerful region-level VLM adept at 3D spatial reasoning.

Visual Whole-Body Control for Legged Loco-Manipulation
Minghuan Liu*,   Zixuan Chen*,  Xuxin Cheng,  Yandong Ji, Ruihan Yang,  Xiaolong Wang

arXiv, 2024
arXiv / Project Page / Code

Visual whole body control with legged robot for mobile manipulation.

Learning Generalizable Feature Fields for Mobile Manipulation
Ri-Zhao Qiu*,   Yafei Hu*,   Ge Yang,   Yuchen Song,   Yang Fu,   Jianglong Ye,   Jiteng Mu, Ruihan Yang,  Nikolay Atanasov,   Sebastian Scherer,  Xiaolong Wang

arXiv, 2024
arXiv / Project Page

Open-ended, Language-grounded Mobile Manipulation with legged robots in the wild with generalizable NeRF.

Expressive Whole-Body Control for Humanoid Robots
Xuxin Cheng*,  Yandong Ji*,  Junming Chen, Ruihan Yang,  Ge Yang,  Xiaolong Wang

RSS, 2024
arXiv / Project Page

Our policy can control a humanoid robot to walk in different styles, shake hands with humans, and even dance with a human in the real world

Harmonic Mobile Manipulation
Ruihan Yang,  Yejin Kim,  Rose Hendrix,  Aniruddha Kembhavi,  Xiaolong Wang,  Kiana Ehsani

IROS, 2024 (Oral)
arXiv / Project Page

Our model finish daily household tasks with minimum sensory data and simplest robot methodlogy.

Generalized Animal Imitator: Agile Locomotion with Versatile Motion Prior
Ruihan Yang*,  Zhuoqun Chen*,  Jianhan Ma*,  Chongyi Zheng*,  Yiyu Chen,
 Quan Nguyen,  Xiaolong Wang

arXiv, 2023
CORL Deployable Workshop, 2023 (Best Paper Award)
arXiv / Project Page

Our system learns a Single Instructable Motion Prior from a diverse reference motion dataset.

Neural Volumetric Memory for Visual Locomotion Control
Ruihan Yang,  Ge Yang,  Xiaolong Wang

CVPR, 2023 (Highlight)
arXiv / Project Page

Utilize 3D structure of the surrounding for visual locomotion over challenging terrain

Learning Vision-Guided Quadrupedal Locomotion End-to-End with Cross-Modal Transformers
Ruihan Yang*,  Minghao Zhang*,  Nicklas Hansen,  Huazhe Xu,  Xiaolong Wang

ICLR, 2022 (Spotlight)
RSS VLRR Workshop, 2021 (Spotlight)
arXiv / Project Page / Code

Proposed LocoTransformer for Visual Locomotion with End2End RL

Vision-Guided Quadrupedal Locomotion in the Wild with Multi-Modal Delay Randomization.
Chieko Sarah Imai*,  Minghao Zhang*,  Yuchen Zhang*,  Marcin Kierebiński,
Ruihan Yang,  Yuzhe Qin,  Xiaolong Wang

IROS, 2022
arXiv / Project Page / Code

We propose Multi-Modal Delay Randomization (MMDR) to address the latency from the control pipeline when training with RL agents.

DexMV: Imitation Learning for Dexterous Manipulation from Human Videos
Yuzhe Qin*,  Yueh-Hua Wu*,  Shaowei Liu,  Hanwen Jiang,
Ruihan Yang,  Yang Fu,  Xiaolong Wang

ECCV, 2022
arXiv / Project Page / Code

We propose a new platform and pipeline, DexMV (Dexterous Manipulation from Videos), for imitation learning to bridge the gap between computer vision and robot learning.

Multi-Task Reinforcement Learning with Soft Modularization
Ruihan Yang,  Huazhe Xu,  Yi Wu,  Xiaolong Wang

NeurIPS, 2020
ICLR BeTR-RL workshop, 2020 (Oral)
arXiv / Project Page / Code

Proposed Soft Modularization for Multi-Task RL to avoid gradient interference between tasks and knowledge Sharing across tasks.

Suphx: Mastering Mahjong with Deep Reinforcement Learning
Junjie Li, Sotetsu Koyamada, Qiwei Ye, Guoqing Liu, Chao Wang,
 Ruihan Yang, Li Zhao, Tao Qin, Tie-Yan Liu, Hsiao-Wuen Hon

arXiv, 2020

Built (Strongest Mahjong AI around the world), now well matched to the top professional human player.

UC San Diego, San Diego, CA

• PhD in Machine Learning • 2021 to Present
• M.S in Computer Science • 2019 to 2021
Nankai University, Tianjin, China

• B.E in Software Engineering • 2015 to 2019
The AI Institute (Boston Dynamics AI Institute), Cambridge, MA

• Research Intern
• Starting - Oct. 2024
Allen Institute for AI, Seattle, WA

• Research Intern
• Jun. 2023 ~ Dec. 2023
Adobe Research, San Jose, CA(Remote)

• Research Intern
• Jun. 2021 ~ Dec.2021
Microsoft Research Asia, Beijing, China

• Research Intern at Machine Learning group
• Mar. 2018 ~ Jun.2019
Professional Service

Conference Reviewer: AAAI, ICRA, ECCV, CVPR, ICCV, NeurIPS, IROS, ICLR, Humanoids

Journal Reviewer: RAL, TRO

Teaching Experience

Math 155A, 3D Computer Graphics, UCSD
20Fall, Teaching Assistant

ECE 176, Introduction to Deep Learning & Applications, UCSD
21Winter / 23Winter, Teaching Assistant

ECE 285, Introduction to Visual Learning, UCSD
24Spring, Teaching Assistant


Soccer: Man-United Fans
Gaming: League
Photograph: Novice
Swimming: Novice
Driving: Learning Manual

Template from this handsome guy...