Voyager: capability accumulates in the skill library, not in the model
“We introduce Voyager, the first LLM-powered embodied lifelong learning agent in Minecraft that continuously explores the world, acquires diverse skills, and makes novel discoveries without human intervention.”
Voyager keeps capability in the skill library, not in the model
Opening
An agent that moves to a new environment or a new task usually starts from scratch, and the usual way to keep what it already learned is to fine-tune the model: costly, and it erodes the abilities the agent already had. That is a real problem for anyone running an agent long-term: today the agent can do the job, next week it has forgotten how, while the user still expects the same quality. Voyager answers by moving accumulation out of the model entirely, so that learning happens in a layer outside it.
What the paper does
Voyager is an agent that learns for life in Minecraft: it explores the world on its own, sets its own goals through an automatic curriculum, stores complex behavior as executable code in a skill library, and improves its own programs through iterative prompting fed by three signals, namely environment feedback, execution errors, and self-verification. It talks to GPT-4 through blackbox queries, with no parameter fine-tuning at all. Capability therefore lives in the skill library: code written once still runs on the next attempt, in a new world, without the earlier abilities being lost. The authors compare Voyager against prior methods on the same Minecraft environment.
Findings
-
Voyager has three key parts: an automatic curriculum to maximize exploration, a skill library of executable code for storing and reusing complex behaviors, and an iterative prompting loop that feeds environment feedback, execution errors, and self-verification back into program improvement. (Voyager consists of three key components: 1) an automatic curriculum that maximizes exploration, 2) an ever-growing skill library of executable code for storing and retrieving complex behaviors, and 3) a new iterative prompting mechanism that incorporates environment feedback, execution errors, and self-verification for program improvement)
-
It calls GPT-4 as an outside service through blackbox queries, which removes the need for parameter fine-tuning, so swapping the model does not lose the skills already accumulated. (Voyager interacts with GPT-4 via blackbox queries, which bypasses the need for model parameter fine-tuning)
-
Against the prior state of the art it collects 3.3x more unique items, travels 2.3x longer distances, and unlocks key tech tree milestones up to 15.3x faster. (It obtains 3.3x more unique items, travels 2.3x longer distances, and unlocks key tech tree milestones up to 15.3x faster than prior SOTA)
Caveat
The authors state no limitation in the abstract. The summary reports results on Minecraft and says the full codebase and prompts are released, but it does not say whether the method transfers to other environments, what a model call costs, or how a large skill library is queried as it grows. Those boundaries are in the full paper, and the abstract’s silence about them is not a claim that they do not exist.
What this means for 5ac
The Hermes skill system is the ever-growing skill library: a skill is written once as an executable procedure and then accumulates in the harness layer instead of living inside the model. Gbrain is the adjacent knowledge store, standing in for how Voyager saves and reuses old skills. Because capability sits outside the model, 5ac can change models at the model-routing layer and keep everything it has built, the same way Voyager calls GPT-4 through blackbox queries without touching a single weight. A completed Kanban card with handoff metadata is also an accumulated skill: the next run reads it and does not have to start over.
Source
- Voyager: An Open-Ended Embodied Agent with Large Language Models — https://arxiv.org/abs/2305.16291