Learn from context.
Demonstrations, instructions, and feedback guide a new task. Neural parameters stay fixed; contextual evidence directs the robot’s existing skills.
Foundations · Section 2 ↗Robots you can teach.
Adapt to new tasks through context.
From demonstrations to action.
Understand how robots use context to act, transfer, and learn from experience.
Abstract
arXiv:2609.36012
General-purpose robots must infer what a new task requires and translate that understanding into appropriate physical action. In-context learning (ICL) for robots supports this process by using demonstrations and interaction to direct existing competence with neural parameters held fixed during deployment. We organize this literature review around the interfaces connecting contextual evidence to execution, distinguishing four families: context-conditioned policies, geometric demonstration transfer, world-model-based control, and skill- and agent-based execution. Comparing these interfaces clarifies their transfer assumptions and the roles of training, correspondence, and memory in making context useful. Across manipulation and navigation, we examine how these mechanisms preserve taught requirements as objects, environments, and execution conditions change. This analysis links method design to evaluation practices that distinguish responsiveness to teaching, physical transfer, and benefits from retained experience. The agenda connects compositional transfer to active coaching and physical recursive self-improvement, where targeted teaching and verified lessons improve acquisition of subsequent tasks.
Understanding Robot ICL
Adapt without fine-tuning.
Fine-tuning changes neural weights. Robot ICL changes the evidence available at inference: demonstrations, memory, motion references, or executable procedures.
Adaptation regimes · Section 2 ↗Measure what transfers.
Vary teaching to test what was learned. Vary objects and scenes to test transfer. Compare later attempts to measure the benefit of retained experience.
Evaluation · Section 7 ↗How context shapes action.
Context-conditioned policies
Action inference
Demonstrations and history condition action inference. Transfer depends on preserving task requirements and mapping them to the robot’s controls.
Geometric demonstration transfer
Motion references
Spatial correspondence turns a demonstration into motion references. A controller tracks them; perception and contact changes determine transfer quality.
World-model-based control
Predicted futures
Context predicts future observations or trajectories. Planning converts those predictions into actions that must remain physically achievable.
Skill- and agent-based execution
Executable procedures
Context composes skills, programs, and tools. Execution feedback checks progress and guides recovery when a step fails.
Learn. Transfer. Improve.
From learning a task to transferring it—and using experience to learn the next one better.
A new task.
Identify which order, relation, or procedure is learned from teaching, and which motor skills were already available.
A new situation.
Preserve the taught requirement while adapting its physical realization to different objects, scenes, bodies, or dynamics.
The next task.
Test whether retained experience improves subsequent learning. Distinguish contextual reuse, revised procedures, and neural updates.
Citation & resources
Haojian Huang et al.arXiv:2609.360122026
@article{huang2026roboticl,
title = {In-Context Learning for Robots: Methods and Applications},
author = {Huang, Haojian and
Li, Zexi and
Guo, Junhao and
Zhang, Yehang and
Peng, Wenxuan and
Zhou, Bohan and
Ruan, Weilin and
Wu, Leyi and
Wang, Chenxu and
Su, Jianchong and
Xie, Binghui and
Chen, Wosong and
Xu, Yingjie and
Zhou, Tianhao and
Chen, Suzeyu and
Zhao, Pukun and
He, Jiaqi and
Li, Xinyi and
Li, Runze and
Dong, Peiran and
Dang, Shaoxiang and
Huang, Jing and
Chen, Yingbing and
Chang, Yifan and
Zhang, Tianyi and
Deng, Shiyuan and
Wang, Haozhi and
Wei, Yangkai and
Li, Wenqian and
Yang, Han and
Zhou, Kaiwen and
Liu, Huaping and
Cheng, James and
Shao, Rui and
Wang, Donglin and
Jin, Yaochu and
Hao, Jianye and
Chen, Ying-Cong and
Li, Yinchuan},
journal = {arXiv preprint arXiv:2609.36012},
year = {2026},
eprint = {2609.36012},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2609.36012}
}