I did a thing. I wasn't planning to do it. It just happened.

In meeting after meeting, a co-worker was showing off his AI combat simulator where he could pit different LLMs (large language models, the technology behind ChatGPT and its relatives) against each other. It had elements of Risk in that it was a capture the map type game, but naval. I was enamored. I had to try one of my own.

His was an RTS, a real-time strategy game, in a sense, but in it you'd actually watch playbacks instead of the fight itself in, as the type of game implies, real time. At first I believe it to be a quirk of his implementation. It wasn't. I’d understand why later, and it would change my plans as a builder dramatically.

What I decided to build was a simulator in the spirit of World of Warcraft, where LLMs could face challenges against opponents that are either predictable or not. After a few week grind, I decided to call it WHOA, or Winning Hinges on Abilities.

Metrics matter; always have and will

For anyone who never played it, World of Warcraft is a game where you build a character, pick a role, and fight monsters. Possibly with friends. A tank absorbs the damage and keeps the enemy's attention on itself. A healer keeps everyone else alive. The others are responsible for DPS, or damage per second, which is both a measurement and a job title. ”The recipe” for group encounters was five players, one tank, one healer, three DPS. Or three to stay lean (but they need to be strong). I’m sure you can guess which three.

I was pretty heavily into WoW years back. In fact, I was a top 100 ranked healer at the Sindragosa encounter. And by Top 100, I mean 99th. But still, with 12 million people playing the game at one point, I’ll take it.

This was back when a band of nerds called Elitist Jerks was a thing. If you weren't there, picture a forum where the smartest people playing the game spent their free time reverse-engineering combat math. They measured abilities, built spreadsheets, and published ability rotations that the rest of us copied without question. I even wrote my own rotation sim in Perl back then to get every last bit of DPS from my Worgen Hunter and his pet.

No shame; I got to the point that I was soloing 5-man dungeons, and when I couldn't do that for reasons other than mechanics, I used a technique called “dual-boxing” to have my ranked healer follow along and boost my survivability. I still have some fun videos of that.

Enter(ing) the Gauntlet

So seeing my co-worker’s naval combat demo got me thinking. Could an LLM out-play a person in such variable games? Could it figure out the abilities and rotations? Could I write deterministic bots that could outplay agents?

I started building. Because of course I did. I was doing everything right; planning implementations and classes, being sure to DRY, and documenting as I went. No, I didn’t plan first. Should I have? Probably, but, at least from time to time, where’s the fun in that?

I got maybe a third of the way into the first question before realizing I'd skipped a prerequisite. I had no reliable way to tell whether anything in my game was playing well. Not an LLM agent, not a deterministic bot, not even me. So that's where the work actually started, just two days in.

The Dummy Never Hits Back (the Data Does)

I mentioned DPS earlier, or damage per second. It’s perhaps the most important metric in this type of game. If you’re doing 60 DPS against an opponent with 600 health it should take 10 seconds to take the win. To give some perspective on WHOA, I targeted 30 DPS for single opponents with about 1000 health, so roughly 30 seconds standing still. Tanks hit softer but have more health (e.g. they “tank” the damage), so 15 DPS against 1500 health (100 seconds).

To test this I created a “target dummy”, that was unbeatable, and a “bot” client to attack it. The bot had access to different abilities (punch, kick, slap, whatever), each with different attack speeds (how quickly it can be re-used) could be repeated serially or mixed in rotations to find max DPS.

Between different tanks I looked to optimize for 15 DPS. For damage dealers I targeted 30 DPS. I did this by giving them four abilities; two basic and two advanced, two slow and two fast. Each did differing damage per attack and required a variable amount of time to become available.

I was stoked; I built a combat engine and now I could test all the different character classes/roles and their abilities. What I didn’t expect was that I, not the code, would fail that test!

Where I failed was in multi-opponent encounters. Remember the tank/dps/healer combo? When I put two of those against each other I saw numbers all over the place. The special abilities, like damage-over-time (DOTs) or multi-target (area-of-effect, or AOE) was totally imbalanced. And I was rapidly becoming unhinged.

I tried 3v3 encounters, but the healers always died first, I never got good data, so I had to pivot. Tuned things a bit, went with a multi-healer 5v5 config (one single-target, one full-party), tried multiple target dummies, and, and, and. Ultimately I had to make a choice; I went with the most “real world” I could; every possible one-per-role five-person party against a mob encounter, every combination rather than a sample. Aggregated.

That back-end balance was a humbling thing to learn from this. Studios tune balance constantly, patch after patch, and players like me used to treat every change as either genius or malice. Elitist Jerks had the same tooling problem: everyone's sim was their own, nobody's agreed with anybody else's.

Round One: The Agents Win

Next I built agents. Agents are players, same as a bot, but are commanded by AI (a LLM, like Claude or ChatGPT). When I first put agents and bots in the same fight, the bots had exactly one kind of intelligence: priority rotations. A priority rotation is a ranked list of "if you can do this, do it, otherwise try the next thing," which is more or less how most WoW players actually play, and producing those lists is what Elitist Jerks existed to do.

I built mine the way I build anything now. I theorycraft first, which is a term for reasoning your way to an answer from mechanics, then I sweep, running the parameters across their range to see how the answer moves. Finally I run the numbers as I mentioned above.

I cracked a can of pringles as I set the first agent-vs-bot fight. Then another. Then another. Much to my surprise, the agents won every time, despite having lower DPS. Say what??

There’s another mechanic that played into the special abilities I mentioned earlier: crowd control. Stuns (no action for 2 seconds), silences (no spell casting, like heals), disarms (no melee), these were making all the difference. The AI knew the abilities existed and used them, locking up their opponents and receiving almost no challenge in response.

Round Two: Not to Interrupt, but...

So I taught them, and the first thing it cost me was a bug that had been sitting in the open for uncounted days. Allow me to summarize: witches had a 12 tick (4 ticks per second, so 3 second) silence, and it was on a 12 tick cooldown. Rightfully, this was abused the first time I fired up a witch bot. I smiled as I fixed it. “Never f with a witch”, I laughed, referencing way too many characters, like Wanda in the MCU.

Teaching CC took more than a line of code. It took a pile of judgment. When is a stun worth a global cooldown you could have spent on damage? At what enemy health does that math flip? How many hostiles have to be in the fight before locking one down beats hitting all of them? Every one of those is a threshold, and every threshold needed tuning against the only instrument I trust.

With CC in their lists and thresholds tuned, the bots beat the agents again. Spoiler: they still do.

Round Three: Teamwork makes...

Then it got harder, because the next thing bots needed wasn't a better rotation. It was each other.

When does a healer share heals across the party instead of topping off one target? When should a player eat, meaning spend their own action self-healing instead of contributing? At what point does a healer abandon whoever it's currently keeping alive and move to someone closer to dying? None of that is throughput, it’s strategy. How do you program strategy? This is where LLMs should dominate... but that’s not to say we can’t try.

Some of the work was mechanical. Real MMOs run on threat, sometimes called aggro: every action generates a score, and enemies attack whoever's score is highest. That's the entire reason tanks exist, and it's the reason a tank is durable at all. My bots had no such instinct, so I forced it, then went back and shuffled the ordering so that additional spawned enemies, the adds, behaved in a way that was interesting to fight instead of piling onto one target like a pack of dogs.

The closest thing to strategy was a simple rule like “if you’re below 25% and have a heal, use it”. I added a “if you have a teammate below 20% and a heal available, use it”. You get the idea. Those applied to all classes, the healers themselves were a bit more complicated. And it worked. Beautifully.

Who’s on First

One of the common things in multi-player engagements in MMOs is the targeting strategy. With a full environment that involves movement (ours does not) there’s a natural order: tanks in front, damage in the middle, healers at the back. Because we don’t have that here (yet) everyone is effectively on top of each other and therefore there are no mechanics preventing a full-tilt assault on the healers. And that’s what the LLMs do every time.

To be fair, in server PVP, killing the healer is the fastest way to win. Focus the squishiest thing, remove the thing keeping the others alive, and the fight is over. An agent doing that has almost certainly absorbed a great deal of human writing about how to actually win at this game, and it is applying it.

While the bots have a progression that matches this paradigm (tanks, melee, ranged, heals), the agents don’t. Which makes combat interesting. Doesn’t matter if you tell them who to attack in the prompt or not, they gravitate to the lowest health (typically the healer) almost every time.

Make It So

Regardless of attack order, the bots win now. After three weeks building the engine that will allow players, bots, and AI to sit at the same table and play the same game; after two weeks tweaking and tuning bots to be the most viable deterministic combatants possible. And after a week of optimizing prompts to cache organically, building team communication and agent journals for memory, I turned back to…

Tuning. Wah wah.

So I froze the code. I’m tired of using Termius on my iPad to SSH into my m5’s tmux session to tweak and tune and await outcomes. You know what we need? That’s right. The cloud! Infinite matches running at the same time, every combination of every ability, aggregation of mountains of data vs the shovelfuls on my laptop. What’s that? Why the iPad when I have a laptop? It’s lighter, cooler, and quieter. Plus, touch screen. Exactly.

For starters, we needed a base of operations. A web server, that hosted all our documentation, allowed us to tweak abilities, and to visualize combat. Like my co-worker did; remember that from the start?

From the web server we’d need to spawn combat servers. Each of those combat servers needs to be reachable and have its own event stream.

Those event streams need to be reachable by a human, a bot, or an AI agent. Remember how we paid attention to structure at the beginning? Bonus: these all use the same code to interact with the server, they just have different front-ends (keyboards, algorithms, or LLMs).

What do you do when you have something that needs 1..n copies of something, and each of those somethings 1..n more somethings attached to them? That’s right: containers!!

For those of you unaware, I’m working at Digital Ocean now. So I launched a managed Kubernetes cluster (DOKS).

First to be added: web servers. Yes, plural. As S.R. Hadden said in Carl Sagan’s Contact, “why build one when you can have two for twice the price?”. So twice free is free. It also allowed me to use Spaces (DigitalOcean’s object storage) via the JuiceFS FUSE adapter to manage assets.

Combat servers were next. They have the same JuiceFS file system to store logs, from which the web server can display fights.

All of these sit behind a load balancer that’s static configured to open 10 ports: one for the web server and nine as persistent instantly-available ad-hoc ports for server endpoints. I love k8s to a point, and dynamic reconfiguration of the network is one of those “to a points”.

The obvious next addition is the spawning of bots, and of course that’s done as containers, as well. They have no file system, though; they pull their content (abilities) from the web server via APIs. Once they have what they need the “JOIN” the combat server.

AI agents were last. I needed to prove all of the above first. To be fair it took a couple days as I couldn’t blow off meetings to build a toy, no matter how shiny it was going to be.

What I didn’t like about the full solution, and there’s no easy fix or it, was scaling speed. One or two combatants; no problem. Eight? Nodes needed to launch and that took time. I was about to put them on functions, but considering each tick (remember, four ticks per second, aka 4 Hz) waiting for an agent to respond could be 30s or a minute (even more when unlucky), so there’s no way they’d complete a fight in the 15m time limit.

MicroVMs, currently in Private Preview, are like long-running functions, and considering i have the load balance set up they’re still viable from a connectivity standpoint, but I didn’t want to have to build and launch one every time.

Another Private Preview service, however, hit the nail on the head. Managed Agent Runtime Services (MARS) allows a single config to be defined in a single YAML file. I can invoke that, have it callback to the server, pick up the assets it needs, join the stream, and run however I want it to run. Maybe it’s a witch on the red team or a brute on the blue team. Doesn’t matter. And I can launch as many as I want at a time.

In the End

I won’t bore you with implementation details of all of this; you see the architecture, both software and infrastructure. Suffice it to say, that the first real test of the system, a 4v4 LLMvLLM matchup that lasted the full 300 seconds the server allows forcing a draw, took a full 8.5 hours overnight, and the only evidence left when it was done was some drops in an object storage bucket.

8.5 hours? No wonder my co-worker’s project was replayed and not real-time. I’d have needed a lot more Pringles!

Every LLM behaved differently; some well, some not so much. GLM, DeepSeek, Kimi, and different versions across each. I had them fight my bots and I had them fight each other. Perhaps the most impressive part was launching three 4v4s at the same time, so 24 agent connections through MARS off the single config. The model works. Wait, I mean the design, not the LLM.

As for answering the questions: could I beat an LLM, could it learn techniques and abilities, and could my strategy-infused bots beat an LLM? Well... it depends. Let's just say the conclusions could not be reached objectively.

Still, when you look at the logs and view the playback, the project as a whole was, in a word… flawless