Hangxin Liu

I am a research scientist and team lead of the Robotics Lab at Beijing Institute for General Artificial Intelligence. In the robotics lab, my colleagues and I hypothesize that there exists some fundamental representations or cognitive architectures underpinning the intelligent behaviors of humans. By uncovering and reproducing such an architecture, we hope to enable long-term human-robot shared autonomy by bridging

• Perception (how to extract and organize more expressive symbols from dense sensory signals);
• Reasoning (how to utilze thoes abstract information for higher-level skills, which in turn faciliates perception);
• Task and Motion Planning (how to effectively act and react).

I received a Ph.D. in Computer Science and a M.S. in Mechanical Engineering from UCLA, working in the Center for Vision, Cognition, Learning, and Autonomy with Professor Song-Chun Zhu. My work at UCLA was supported by DARPA SIMPLEX, DARPA XAI, ONR MURI, and ONR Cognitive Robot.

Before joining VCLA, I graduated with a B.S. in Mechanical Engineering and a B.S. in Computer Science with a Mathematics minor from Virginia Polytechnic Institute and State University (Virginia Tech) in 2016.

CV  /  Google Scholar

I'm interested in robot perception, planning, learning, human-robot interaction, and virtual and augmented reality. Representative papers are highlighted.

We extend our work in IJCV22 and ICRA21 by segmenting the objects in the panoptic map into parts and replace those parts by primitive shapes, which results in more realistic functionally equivalent scenes.

An attributed stochastic grammar is proposed to model the process of object fragmentation during cutting, which abstracts the spatial arrangement of fragments as node variables and captures the causality of cutting actions based on the fragmentation of parts.

We present a novel modular robot system capable of self-reconfiguration and achieving omnidirectional movements through magnetic docking for collaborative object transportation. Each robot in the system only equips a steerable omni wheel for navigation.

Instead of one-step aerial manipulation tasks, we investigate the sequential manipulation planning problem of UAMs, which requires coordinated motions of the vehicle’s floating base, the manipulator, and the object being manipulated over a long horizon.

We present an optimization framework to redesign an indoor scene by rearranging the furniture within it, which maximizes free space for service robots to operate while preserving human's preference for scene layout.

We rethink the problem of scene reconstruction from an embodied agent’s perspective. The objects within a reconstructed scene are segmented and replaced by part-based articulated CAD models to provide actionable information and afford finer-grained robot interactions.

We devise a 3D scene graph representation to abstract scene layouts with succinct geometric information and valid robot-scene interactions, such that a valid task plan can be computed using graph editing distance between the initial and the final scene graph while effectively satisfying constraints in motion level.

Leveraging the input redundancy in over-actuated UAVs, we tackle downwash effects between propellers with a novel control allocation framework that explores the entire allocation space for an optimal solution that reduces counteractions of airflows.

Learning key physical properties of tool-uses from a FEM-based simulation and enacting those properties via an optimal control-based motion planning scheme to produce tool-use strategies drastically different from observations, but with the least joint efforts.

A cooperative planning framework to generate optimal trajectories for a robot duo tethered by a flexible net to gather scattered objects spread in a large area. Implemented Model Reference Adaptive Control (MRAC) to handle unknown dynamics of carried payloads.

Given an interpretable And-Or-Graph knowledge representation, the proposed AR interface allows users to intuitively understand and supervise robot's behaviors, and interactively teach the robot with new actions.

Constructing a Virtual Kinematic Chain (VKC) that readily consolidates the kinematics of the mobile base, the arm, and the object to be manipulated in mobile manipulations.

The VKC perspective is a simple yet effective method to improve task planning efficacy for mobile manipulation.

Reconstructing an interactive scene from RGB-D data stream by panoptic mapping and organizing object affordance and contextual relations by a graph-based scene representation.

Physical robots can control and alter virtual objects in AR as an active agent and proactively interact with human agents, instead of purely passively executing received commands.

A modular robot system that allows non-expert users to build a multi-legged robot in various morphologies using a set of building blocks with sensors and actuators embedded.

A graphical model to unify the representation of object states, robot knowledge, and human (false-)beliefs.

An AR-based indoor evacuation system with a congestion-aware routing solution.

A comprehensive review on cognitive AI and visual commonsense (Functionality, Physics, Intension, Causality).

Learning multi-modal knowledge representation and fostering human trusts by producing explanations.

A testbed with fine-grained physical effects and realistic human-robot interactions for training embodied agents.

Robot 'labels' received data by its own exploration and refines its predictive model on-the-fly.

A data glove for natural human grasping activities in VR and for cost-effective grasping data collections.

Learning functionally equivalent actions that produce similar effects instead of learning trajectory cues.

An AR system for users to diagnose robor's problems, correct wrong behaviors, and add the corrections to the robot's knowledge.

An unsupervised learning approach for manipulation event segmentation and parsing using hand gestures and forces data.

A easy-to-replicate glove-based system for real time collections of hand pose and force during fine manipulative actions.

Learning an action planner through both a top-down stochastic grammar model (And-Or graph) and a bottom-up discriminative model from the observed poses and forces

Design and source code from Jon Barron's website