Robot ICL Survey

Vision-language navigation and spatial memory

A route demonstration, a scene preview and a failure note teach different things. The Robot ICL Survey connects these information needs to vision-language navigation and spatial adaptation. This companion guide explains how to choose the evidence a navigator needs and test whether it changes execution.

Four studies show how evidence reaches motion.

  • Robust Path Following (RPF) encodes demonstrated views and actions in a path memory. The policy attends to that representation during noisy execution. Its teaching signal is the demonstrated route, so matching current progress matters even when camera poses differ.
  • ReLIC retains embodied interaction history across episodes with different goals in one home. The deployed policy stays fixed while experience supplies spatial knowledge. Reusing a familiar environment and acquiring knowledge of a new building are distinct evaluation conditions.
  • LM-Nav uses examples to prompt instruction-to-landmark translation, grounds the landmark sequence in a graph, and delegates motion to a pretrained navigator. It illustrates how language interpretation can use existing perception and control capabilities to execute physical routes.
  • CMMR-VLN retrieves successful trajectories and first-error notes as navigation guidance. The retained outcome explains what should change on another attempt. Its contribution should be separated from simply allowing more backtracking or additional chances to reach the destination.

Test route adherence, goal attainment and stopping separately.

Change the demonstration while keeping the destination fixed to test route teaching. A shortcut may improve path efficiency while violating the prescribed route. Remove a preview or cross-goal memory to test spatial knowledge, and replace instruction examples under the same query to test interpretation. For outcome memory, withhold failure notes while preserving retries and recovery actions.

Report goal attainment, route or landmark-order adherence, wrong turns and stopping errors. Success weighted by path length (SPL) captures attainment and efficiency, but does not by itself establish compliance with a taught path. Match visibility, map access, action discretization and stopping distance. Count exploration, clarification and failed attempts in the total interaction budget.

Test new goals in a known environment separately from navigation in unseen buildings. Record whether evidence comes from simulation, recorded waypoint assessment or physical execution; these settings establish different transfer claims. The survey develops these controls in context use and transfer evaluation.

Make spatial memory responsive to change.

A new goal can reuse a map while replacing the route. Moving an object invalidates its location without necessarily invalidating the room layout. A useful research direction is selective updating that retains observation time, provenance and the conditions under which a place or object was visible. Absence from one image is weaker evidence when the expected location was occluded.

Navigation also needs a path from revised beliefs to active commands. After a failed passage, updating a plan should change the pending movement or stopping decision. Tests can pair correct spatial evidence with controlled execution interfaces to locate whether failure comes from retrieval, interpretation, route selection or motion realization.

Continue through the survey.

Read navigation and spatial adaptation, navigation context types, shared memory mechanisms, and the evaluation controls and resources.

Related topics: memory for embodied agents, vision-language-action models, and robot world models.