Identify the object that improves.
A successful retry can follow from a better prompt, additional physical progress, a repaired program, or a stronger controller. These explanations imply different costs and transfer conditions. The survey separates contextual and parameter adaptation and tracks the artifacts retained between attempts.
| Update route | What changes | A useful control |
|---|---|---|
| Contextual reuse | Observations, retrieved episodes or correction guidance supplied to a fixed model. | Remove the retained evidence while preserving the scene, executor and retry budget. |
| Program or skill revision | Executable code, skill composition, settings or the procedure constructing context. | Restore the earlier artifact and test both repaired and previously successful tasks. |
| Training and consolidation | Policy or language-model parameters learned from collected experience. | Compare checkpoints on later tasks under matched teaching and execution conditions. |
| Test-time parameter adaptation | Fast weights or another neural subset updated during deployment. | Reset adaptive parameters separately from explicit history; include optimization cost. |
Persistence is a separate design choice. A memory can survive multiple attempts without changing weights; a parameter update may be discarded at an episode boundary. Reporting the updated component and reset rule makes “improvement” a testable claim.
Four studies locate distinct update loops.
- DROC converts language corrections into reusable textual guidance. Later retrieval can change plans and skill calls with the language model fixed. This illustrates how retaining an applicable constraint differs from acquiring new motor competence.
- RoboRSI associates failures with branches of a task–skill hierarchy, scopes revisions to those branches, and validates changes against earlier executions. Stable sequences become parameterized routines. Its code-reuse mechanism is distinct from its separate policy-training branch.
- Language Model Predictive Control (LMPC) uses instructions and corrections to revise reward code within a teaching session. Across sessions, successful exchanges train the language model. The later teaching interaction therefore becomes an evaluation target, linking immediate feedback use with learned teachability.
- RoboTTT uses sequential weight updates to encode video and rollout history. The survey places it among parameter-adaptation comparisons: testing it requires distinguishing stored observations from the adaptive neural state carrying their effect.
Separate practice from later capability.
First evaluate reuse on the practiced task with a physical reset and equal attempts. Then test changed objects, layouts and skill compositions. Keep the initial executor, observation access and interaction budget matched when attributing gains to retained artifacts. Freeze acquired artifacts before held-out evaluation when the question is transfer from completed practice.
Report per-attempt success, repeated failures, adherence to corrections, human interventions and total acquisition cost. Cumulative coverage also increases when a system gets more chances; it does not establish reliable reuse. Regression tests should revisit previously solved tasks after each accepted skill or parameter change.
Verification deserves its own intervention. A system may produce better-looking explanations while accepting ineffective edits. Compare proposed revisions with independently checked outcomes, and record rejected changes as part of the learning cost. The survey’s controlled attribution framework connects these tests to the component that changed.
Test whether the learner becomes easier to teach.
Recursive self-improvement is a stronger hypothesis than success after local repair: a revised system should acquire or verify subsequent tasks more effectively while preserving earlier capabilities. One research design evaluates successive learners on new procedures with informative, absent and meaning-swapped teaching. This separates improved use of feedback from a stronger prior policy.
A related opportunity is selecting the next intervention from the diagnosed failure. Missing task information may call for clarification; unreliable contact may require a probe or motor practice. Measure whether choosing among contextual guidance, program revision and parameter learning reduces total teaching and trials. Retaining the conditions under which a repair works is essential to making that choice reusable.
Continue through the survey.
Read external knowledge reuse and revision, external artifact roles, and active coaching and physical recursive self-improvement, with its comparison of update mechanisms.
Related topics: memory for embodied agents, learning from demonstration, and robot world models.