Context Juggling and Multi-Agent Workflows

Lately, I have found myself incorporating multi-agent workflows into my daily work. At the same time, published evaluations usually find that teams of agents are not worth it. For the kind of problem that a proper evaluation can measure, that makes sense to me. The vibes I feel daily are different, and for daily work, that makes sense too. So it seems timely to share my experience with these workflows. At the pace things are moving, I think even vibes can be interesting to other people.

I have been running several agents in parallel, on different projects as well as on the same project at once. For a while now, it has been popular to have different models review each other’s work, often adversarially. But I wanted to see whether I could get the available models to cooperate in the swarmy way the OpenAI agents displayed in the Hugging Face incident. I am as concerned as the next guy, and at the same time in awe of that display of potential and skill.

There are two caveats straight off the bat. First, the frontier models available in coding agents, like Opus and Fable, have not been trained to cooperate, so simply telling them to work together is not going to go perfectly. Second, I don’t have OpenAI’s resources to waste on my comparatively simple problems.

I still wanted to try it, and I have been using it a lot lately. At first, I set up proper swarms. They were small, but I enjoyed calling them swarms. I watched how the agents played with each other and worked out the problems. As usual with these things, the result was a set of instructions I give the agents when they start, so that they cooperate usefully. I no longer use the small test swarms, because the overhead didn’t feel appropriate. But I have used agentic cooperation regularly ever since, and my vibes say it works well.

What the evaluations measure

There are small evals on this by now, and I notice more and more of them. The groups behind them usually find that having multiple agents cooperate on a problem is not worthwhile. The setup tends to be the same: a problem whose solution can be checked as right or wrong, solved either by one agent alone or by a team of two, four or eight agents. Each variant runs many times, and the evaluation compares how often it succeeded and what it cost in tokens and time. The outcome is usually that the team is not worth it, and a single agent doing its thing is more efficient.

My daily work looks nothing like that. Usually, I have agents doing development or research work. With only a single agent, the familiar problems show up: it goes off on side quests, digs into one rabbit hole, or gets caught up in details and wants my decisions on things that aren’t critical to the overall effort. And it forgets stuff.

Context juggling

I have not written a line of code by hand in quite some time. What I do instead no longer feels like prompt engineering, harness engineering, or even context engineering. It feels like what I would call context juggling. My job is to help the agents keep their context as useful as possible, with the right information in it and as little noise as possible.

Context engineering, as I understand it, has mostly been about single-agent workflows. I still do plenty of it, going back and forth: one minute I fill in information the context is missing, the next I prune it to improve the signal-to-noise ratio. With multi-agent setups, it feels even more like juggling. It’s not as technical a term, but it describes what I’m doing.

How my agents work together

For context juggling, multi-agent setups are a really useful tool. There is the development agent, which works on whatever issue or feature is at hand. I usually let it spawn subagents freely, because subagents keep its context clean and streamlined. Then come the helper agents: usually one concerned with infrastructure or hardware, one with product direction, and a pacemaker that makes sure budgets are kept and nobody wanders off on side quests.

Only one agent writes. Which agent that is can change over time, while the team itself stays the same. At one point, the team might focus on something that needs to be written, and at another on analysis.

What makes this work is one important instruction: the agents deliberate before major steps or before moving on to something else, and they take any problem they run into to the group. The group can always weigh in, and decisions need consensus. That rule is simple enough to get them talking to each other, and it lets each agent focus on what it should be focusing on. The risk that they chitchat too much and get distracted has, for me, never materialised. One agent keeping the others on track seems much more promising to me than refining a single prompt or harness until the development agent also paces itself properly.

My agents don’t necessarily solve a given, well-defined problem together. They cooperate to keep each other’s contexts cleaner and leaner, and on track. Their collaboration is context juggling, which is exactly the part of my job that the multi-agent setup helps with so much. I would love to find evaluations that capture this, though I think it is harder to do in vitro.