Ruihan Yang

I'm currently a first second third fourth-year PhD candidate at UC San Diego, advised by Prof Xiaolong Wang. Before coming to UC San Diego, I received my B.E. in Software Engineering from Nankai University in 2019

I'm interested in reinforcement learning, machine learning, robotics and some system stuff. Specifically, I'd like to build intelligent agent, which make decision with information coming from different sources

Email  /  CV  /  Google Scholar  /  Github  /  Linkedin

I'm open to discussion or collaboration. Feel free to drop me an email if you're interested in my research.

profile photo
Publication & Preprint
* denotes equal contribution

Manipulation

Visual Whole-Body Control for Legged Loco-Manipulation
Minghuan Liu*,   Zixuan Chen*,  Xuxin Cheng,  Yandong Ji, Ruihan Yang,  Xiaolong Wang

CORL, 2024 (Oral)
arXiv / Project Page / Code

Visual whole body control with legged robot for mobile manipulation.

Harmonic Mobile Manipulation
Ruihan Yang,  Yejin Kim,  Rose Hendrix,  Aniruddha Kembhavi,  Xiaolong Wang,  Kiana Ehsani

IROS, 2024 ( Best Paper on Mobile Manipulation / Oral)
arXiv / Project Page

Our model finish daily household tasks with minimum sensory data and simplest robot methodlogy.

ACE: A Cross-Platform Visual-Exoskeletons System for Low-Cost Dexterous Teleoperation.
Shiqi Yang,  Minghuan Liu,  Yuzhe Qin,  Runyu Ding,  Jialong Li,  Xuxin Cheng,  Ruihan Yang,  Sha Yi,  Xiaolong Wang

CORL, 2024
arXiv / Project Page / Code

Our (low-cost cross-platform visual-exoskeleton) teleoperation system can generalize to humanoid hands, arm-hands, arm-gripper, and quadruped-gripper systems with high-precision teleoperation.

Bunny-VisionPro: Real-Time Bimanual Dexterous Teleoperation for Imitation Learning
Runyu Ding*,  Yuzhe Qin*,  Jiyue Zhu*,  Chengzhe Jia,  Shiqi Yang, Ruihan Yang,  Xiaojuan Qi,  Xiaolong Wang

arXiv, 2024
arXiv / Project Page / Code

Bunny-VisionPro is a real-time bimanual teleoperation system that prioritizes safety and minimal delay. It also features human haptic feedback to enhance immersion.

Visual Manipulation with Legs
Xialin He*,  Chengjing Yuan*,  Wenxuan Zhou,  Ruihan Yang,  David Held,  Xiaolong Wang

CORL, 2024
arXiv / Project Page

We propose a system that enables quadruped to manipulate objects with legs.

Learning Generalizable Feature Fields for Mobile Manipulation
Ri-Zhao Qiu*,   Yafei Hu*,   Ge Yang,   Yuchen Song,   Yang Fu,   Jianglong Ye,   Jiteng Mu, Ruihan Yang,  Nikolay Atanasov,   Sebastian Scherer,  Xiaolong Wang

arXiv, 2024
arXiv / Project Page

Open-ended, Language-grounded Mobile Manipulation with legged robots in the wild with generalizable NeRF.

DexMV: Imitation Learning for Dexterous Manipulation from Human Videos
Yuzhe Qin*,  Yueh-Hua Wu*,  Shaowei Liu,  Hanwen Jiang,
Ruihan Yang,  Yang Fu,  Xiaolong Wang

ECCV, 2022
arXiv / Project Page / Code

We propose a new platform and pipeline, DexMV (Dexterous Manipulation from Videos), for imitation learning to bridge the gap between computer vision and robot learning.

Locomotion

Expressive Whole-Body Control for Humanoid Robots
Xuxin Cheng*,  Yandong Ji*,  Junming Chen, Ruihan Yang,  Ge Yang,  Xiaolong Wang

RSS, 2024
arXiv / Project Page

Our policy can control a humanoid robot to walk in different styles, shake hands with humans, and even dance with a human in the real world

Generalized Animal Imitator: Agile Locomotion with Versatile Motion Prior
Ruihan Yang*,  Zhuoqun Chen*,  Jianhan Ma*,  Chongyi Zheng*,  Yiyu Chen,
 Quan Nguyen,  Xiaolong Wang

CORL, 2024
CORL Deployable Workshop, 2023 (Best Paper Award)
arXiv / Project Page

Our system learns a Single Instructable Motion Prior from a diverse reference motion dataset.

Neural Volumetric Memory for Visual Locomotion Control
Ruihan Yang,  Ge Yang,  Xiaolong Wang

CVPR, 2023 (Highlight)
arXiv / Project Page

Utilize 3D structure of the surrounding for visual locomotion over challenging terrain

Learning Vision-Guided Quadrupedal Locomotion End-to-End with Cross-Modal Transformers
Ruihan Yang*,  Minghao Zhang*,  Nicklas Hansen,  Huazhe Xu,  Xiaolong Wang

ICLR, 2022 (Spotlight)
RSS VLRR Workshop, 2021 (Spotlight)
arXiv / Project Page / Code

Proposed LocoTransformer for Visual Locomotion with End2End RL

Vision-Guided Quadrupedal Locomotion in the Wild with Multi-Modal Delay Randomization.
Chieko Sarah Imai*,  Minghao Zhang*,  Yuchen Zhang*,  Marcin Kierebiński,
Ruihan Yang,  Yuzhe Qin,  Xiaolong Wang

IROS, 2022
arXiv / Project Page / Code

We propose Multi-Modal Delay Randomization (MMDR) to address the latency from the control pipeline when training with RL agents.

Perception

SpatialRGPT: Grounded Spatial Reasoning in Vision-Language Models
An-Chieh Cheng,   Hongxu Yin,  Yang Fu,  Qiushan Guo, Ruihan Yang,   Jan Kautz,  Xiaolong Wang,  Sifei Liu

NeurIPS, 2024
arXiv / Project Page

A powerful region-level VLM adept at 3D spatial reasoning.

Reinforcement Learning

Multi-Task Reinforcement Learning with Soft Modularization
Ruihan Yang,  Huazhe Xu,  Yi Wu,  Xiaolong Wang

NeurIPS, 2020
ICLR BeTR-RL workshop, 2020 (Oral)
arXiv / Project Page / Code

Proposed Soft Modularization for Multi-Task RL to avoid gradient interference between tasks and knowledge Sharing across tasks.

Suphx: Mastering Mahjong with Deep Reinforcement Learning
Junjie Li, Sotetsu Koyamada, Qiwei Ye, Guoqing Liu, Chao Wang,
 Ruihan Yang, Li Zhao, Tao Qin, Tie-Yan Liu, Hsiao-Wuen Hon

arXiv, 2020
arXiv

Built (Strongest Mahjong AI around the world), now well matched to the top professional human player.

Education
UC San Diego, San Diego, CA

• PhD in Machine Learning • 2021 to Present
• M.S in Computer Science • 2019 to 2021
Nankai University, Tianjin, China

• B.E in Software Engineering • 2015 to 2019
Experience
Allen Institute for AI, Seattle, WA

• Research Intern
• Jun. 2023 ~ Dec. 2023
Adobe Research, San Jose, CA(Remote)

• Research Intern
• Jun. 2021 ~ Dec.2021
Microsoft Research Asia, Beijing, China

• Research Intern at Machine Learning group
• Mar. 2018 ~ Jun.2019
Honors & Awards

Best Paper on Mobile Manipulation @ IROS 2024 (1 out of 2000+) Oct, 2024
Best Workshop Paper @ CORL 2023 Deployable Workshop Nov, 2023
Professional Service

Conference Reviewer: AAAI, ICRA, ECCV, CVPR, ICCV, NeurIPS, IROS, ICLR, Humanoids

Journal Reviewer: RAL, TRO

Teaching Experience

Math 155A, 3D Computer Graphics, UCSD
20Fall, Teaching Assistant

ECE 176, Introduction to Deep Learning & Applications, UCSD
21Winter / 23Winter, Teaching Assistant

ECE 285, Introduction to Visual Learning, UCSD
24Spring, Teaching Assistant

Misc

Soccer: Man-United Fans
Gaming: League
Photograph: Novice
Swimming: Novice
Hiking
Driving: Learning Manual


Template from this handsome guy...