Environment Dynamics Matters:
Embodied Navigation to Movable Objects

Introduction

---

The embodied navigation, which is a typical kind of embodied cognition behavior, has attracted great attention from researchers from the robotics, machine learning, and computer vision communities. In recent years, a lot of machine learning methods are proposed to tackle the embodied navigation problem. Moreover, many simulation environments have been established, which enable the work that originally requires cumbersome physical verification to be conveniently conducted in the simulator and great progress has been made in terms of relevant methods. It is true that the developed machine learning methods could provide effective approaches for the navigation task, while they are restricted to the paradigm of applying the trained model on test scenes, which is hard to adapt to the dynamic environment. In reality, the environment dynamics is ubiquitous and it often occurs that objects we want to find are movable or have some temporal location patterns associated with user habit routines. It is difficult both for existing simulation environments to nicely simulate environment dynamics and for existing algorithms to take advantage of environment dynamics to improve search efficiency. Fortunately, the AIGC and LLM technologies are developing rapidly and becoming mature, which shed light on the above challenges. Besides, there also emerge a series of great research works which take environment dynamics into consideration. Therefore, we believe it could be a perfect time to have this workshop which provides an opportunity to gather researchers to discuss on the topic and further accelerate the development of the field.

Content of Workshop

---

In this workshop, we aim to emphasize the importance of environment dynamics in the long-horizon embodied navigation and manipulation task. As the environment dynamics is ubiquitous (for example, the same object could have been placed in different places associated with user routines), the robot should consider the movement of the object rather than searching static ones. Given the great challenges brought by the dynamic environment both in the establishment of simulation environment and corresponding algorithms, the tentative content of the workshop is intended to include the following topics.

• extraction of spatio-temporal object information from the long-term interaction with the environment
• modeling of spatial-temporal object information using advanced machine learning technologies such as graphical model or cognitive-inspired method
• generation of dynamic simulation environment using AIGC technologies (LLM, diffusion, etc.)
• planning of smart actions for dynamic embodied navigation and manipulation
• deployment of dynamic embodied navigation and manipulation on real-world robotic platform

Schedule (Tentative)

---

Time	Title	Speaker
14:00 – 14:10	Introduction & Welcome
14:10 – 14:50	Robot Learning for Long-Horizon Mobile Manipulation	Prof. Abhinav Valada University of Freiburg, Germany
14:50 – 15:30	Real-time 4D Metric-Semantic Mapping	Prof. Luca Carlone MIT, USA
15:30 – 16:10	Longitudinal Modeling of Grounded User Routine	Maithili Patel Georgia Institute of Technology, USA
16:10 – 16:30	Coffee break & Poster Session
16:30 – 17:10	Interactive Navigation and Interactive Search: Increasing robots capabilities in dynamic human-centered environments	Prof. Roberto Martin-Martin University of Texas at Austin, USA
17:10 – 17:50	Embodied Intelligence with Foundation Models for Mobile Manipulation	Dr. Oier Mees UC Berkeley, USA
17:50 – 18:10	Oral for Best Paper candidates
18:10	Award Ceremony

Invited Speakers

---

Luca Carlone MIT	Roberto Martin-Martin University of Texas at Austin	Abhinav Valada University of Freiburg
Oier Mees UC Berkeley	Maithili Patel Georgia Institute of Technology

Organizers

---

Huaping Liu Tsinghua University	David Hsu National University of Singapore	Di Guo Beijing University of Posts and Telecommunications
Tao Kong ByteDance Research	Abhishek Gupta University of Washington

Endorsement

---

IEEE RAS Technical Committee on Algorithms for Planning and Control of Robot Motion
IEEE RAS Technical Committee on Cognitve Robotics
IEEE RAS Technical Committee on Machine Learning for Automation
IEEE RAS Technical Committee on Mobile Manipulation
IEEE RAS Technical Committee on Robot Learning
CAA Technical Committee on Intelligent Automation

---

Support

---

Contact

---

For further information or questions, please contact us at edmws2024@gmail.com