Learning from real-world robot demonstrations holds promise for interacting with complex real-world environments. However, the complexity and variability of interaction dynamics often cause purely positional controllers to struggle with contacts or varying payloads. To address this, we propose a Heterogeneous Meta-Control (HMC) framework for Loco-Manipulation that adaptively stitches multiple control modalities: position, impedance, and hybrid force-position. We first introduce an interface, HMC-Controller, for blending actions from different control profiles continuously in the torque space. HMC-Controller facilitates both teleoperation and policy deployment. Then, to learn a robust force-aware policy, we propose HMC-Policy to unify different controllers into a heterogeneous architecture. We adopt a mixture-of-experts style routing to learn from large-scale position-only data and fine-grained force-aware demonstrations. Experiments on a real humanoid robot show over 50% relative improvement vs. baselines on challenging tasks such as compliant table wiping and drawer opening, demonstrating the efficacy of HMC.
We propose HMC-Controllers, a unified control interface that seamlessly blends position, impedance, and hybrid force-position controllers in the torque space. This allows for adaptive control strategies that can handle complex interactions with the environment. HMC-Controllers facilitate both teleoperation and policy deployment, enabling robots to perform contact-rich tasks safely and effectively.
Position controller
For precise free-space tasks
Compliance controller
For forceful & contact-rich tasks
Hybrid force-position controller
For force-aware tasks
Ours: HMC-Controller, long horizon video clip
Ours: HMC-Controller, with stiffness annotation
Comparison: uniformly high stiffness, contact-rich insertion becomes jerky and unstable, often generating excessive force and failing to align properly.
Comparison: uniformly low stiffness, motions lose precision and cannot generate enough force to complete insertion reliably.
Task: Screw-nut insertion and tightening
Ours: HMC-Controller
Comparison: Pure Position controller
Task: Peg-in-Hole Insertion
Ours: HMC-Controller
Comparison: Pure Position controller
Task: Wiping Whiteboard
Task: Opening the door
Task: Arranging the chair
Task: Opening the microwave
Task: Putting on cloth and back-tapping massage
Pretraining stage: We harness abundant positional demonstrations to train the shared transformer trunk and position expert head, thereby embedding a strong positional prior that boosts generalization.
Fine-tuning Stage: All parameters are unfrozen and fine-tuned on a smaller, fine-grained multi-expert dataset. A soft routing network learns to blend outputs from multiple experts, producing smooth and adaptive control policies. (J.S: Joint Space. C.S: Cartesian Space.)
Ours: HMC-Policy
Representative Comparison: ACT (meta) policy
Task: Autonomous drawer opening
Ours: HMC-Policy
Representative Comparison: ACT (meta) policy
Task: Autonomous bottle lifting