Research Project

Manipulation-Feasible Navigation Among Movable Obstacles with Discrete-Contact Pushing

Shaohu Wang, Aiguo Song, Yulong Yuan, Zhongyu Sun, Tianyuan Miao, and Qinjie Ji

The State Key Laboratory of Bioelectronics
Jiangsu Key Laboratory of Robot Perception and Control Technology
School of Instrument Science and Engineering, Southeast University

Project video

The project video is not available at the current media URL.

Abstract

In environments with large movable obstacles, detour-only navigation can be inefficient or even infeasible, while obstacle interaction requires reasoning about navigation benefit, feasible placement, and executable manipulation. We present a hierarchical navigation among movable obstacles (NAMO) framework for mobile manipulators. At the high level, the planner identifies key blocking obstacles from reference paths and searches for relocation plans that jointly satisfy geometric, manipulation, and downstream navigation constraints. When direct relocation is hindered by other movable objects, a large language model (LLM) is selectively invoked to infer auxiliary manipulation dependencies, which are then verified by deterministic geometric planning. To execute the resulting relocation goals, we define discrete contact modes on box surfaces and select contact faces and regions online from position and orientation errors, enabling straight, lateral, and corner pushing through contact switching. A recurrent reinforcement-learning policy coordinates the mobile base and manipulator to track tool center point (TCP) targets while preserving end-effector reachability during sustained pushing. Simulation and real-robot experiments demonstrate feasible navigation–manipulation in detour, single- and multi-obstacle relocation, and non-local dependency scenarios, validating the framework for interactive navigation with large non-graspable obstacles. The open-source project will be released soon.

Framework Overview

Overview diagram of the hierarchical NAMO planning and execution framework.
Overview of the proposed hierarchical NAMO framework. The planning layer evaluates navigation-only and NAMO hypotheses and selects the feasible plan with the lowest total cost. NAMO planning identifies key blockers and searches for manipulation-feasible relocation poses; when direct relocation fails, an LLM is selectively invoked to infer auxiliary obstacles and their manipulation dependencies for replanning. The execution layer maps relocation goals to discrete contact modes and TCP reference targets, which are tracked by a learned whole-body controller to compose straight, lateral, and corner pushing, while navigation-only segments follow the selected collision-free path.

Execution

Pushing Skill Demonstration

Video is not available at the current media URL.

TCP Target Tracking
+

Video is not available at the current media URL.

Discrete Contact Point Planning

Video is not available at the current media URL.

Straight Pushing

Video is not available at the current media URL.

Side Pushing

Video is not available at the current media URL.

Corner Pushing

Video is not available at the current media URL.

Straight Pushing

Video is not available at the current media URL.

Side Pushing

Video is not available at the current media URL.

Corner Pushing

Evaluation

Simulation Experiments

Video is not available at the current media URL.

Scene 1

Video is not available at the current media URL.

Scene 2

Video is not available at the current media URL.

Scene 3

Video is not available at the current media URL.

Scene 4

Video is not available at the current media URL.

Scene 5

Video is not available at the current media URL.

Scene 6

Deployment

Real-World Experiments