How AI Actually Works in Modern Video Games: FSMs to LLMs
Game AI isn't intelligent. It's state machines, search, and scoring arranged to feel like judgment, with real, runnable code from FSMs to local LLM dialogue.
I spent years assuming the soldiers in F.E.A.R. were running something close to real intelligence — flanking through doorways, calling out my position, regrouping when I thinned their numbers. They weren't. They were running a three-state machine bolted to a search algorithm, built by one AI programmer at Monolith in 2005. That gap — between how smart something feels and what's actually computing under the hood — is the entire story of AI in games, and two decades of "next-gen AI" marketing hasn't closed it nearly as much as you'd think. Most game AI isn't intelligent in any meaningful sense. It's rules, search, and scoring functions arranged to feel like judgment. This post walks through what's actually running — with real code you can execute, not just diagrams — from the finite state machine your first platformer boss used, up through the local LLM stack studios are experimenting with for NPC dialogue today.
Finite State Machines: The Baseline
The simplest and still most common form of game AI is the finite state machine (FSM). An enemy exists in one of a few discrete states — idle, patrol, chase, attack, flee — and transitions between them on triggers. Spot the player: switch to chase. Lose sight for five seconds: switch back to patrol. Health under 20%: flee. No memory beyond the current state, no lookahead, no planning — just the current state plus the current inputs.
# guard_ai.py
class GuardAI:
def __init__(self):
self.state = "patrol"
def update(self, can_see_player, health_pct, distance_to_player):
if health_pct < 0.2:
self.state = "flee"
elif can_see_player and distance_to_player < 10:
self.state = "attack"
elif can_see_player:
self.state = "chase"
else:
self.state = "patrol"
return self.state
if __name__ == "__main__":
import argparse
parser = argparse.ArgumentParser()
parser.add_argument("--health", type=float, required=True)
parser.add_argument("--can-see-player", type=lambda s: s.lower() == "true", required=True)
parser.add_argument("--distance", type=float, required=True)
args = parser.parse_args()
guard = GuardAI()
result = guard.update(args.can_see_player, args.health, args.distance)
print(f"state: {result}")Run it from a terminal and you can watch the transitions happen in real time as you vary the inputs:
$ python3 guard_ai.py --health 0.8 --can-see-player true --distance 4
state: attack
$ python3 guard_ai.py --health 0.15 --can-see-player true --distance 4
state: flee
$ python3 guard_ai.py --health 0.8 --can-see-player false --distance 4
state: patrolThat's the entire mental model. Cheap, predictable, and trivial for a designer to tune in a spreadsheet — which is exactly why it's still the default backbone for most shooters and platformers, even in engines that also support fancier systems layered on top.
The limitation shows up the moment the world gets more complex than "can I see the player." A pure FSM has no way to represent why it's in a state, only that it is — so anything requiring lookahead (should I retreat through this door or that one?) needs to be hardcoded as yet another state, and the state count explodes. Half-Life's AI reportedly ran over 80 unique states in its FSM for exactly this reason. That explosion is the problem the next system was built to solve.
We recommend reading Build Your Own Stack-Based Bytecode Virtual Machine in C to continue reading our selection of content. Same underlying idea as the FSM above.
The Case That Actually Planned: F.E.A.R.'s GOAP
F.E.A.R. gets cited constantly as the game that "solved" enemy AI, and it's worth being precise about what it actually changed, because it's not what most people assume. AI programmer Jeff Orkin didn't throw out the finite state machine — F.E.A.R.'s soldiers still run one, with exactly three states: go to, animate, and use smart object. What changed was how the game decided what belonged in those states. Instead of hand-scripting "if flanked, retreat," Orkin built a Goal-Oriented Action Planner (GOAP) — a real-time adaptation of STRIPS, the 1971 Stanford planning system — that searches over a space of possible actions, each with preconditions and effects, to find a sequence that satisfies a goal. It uses A* to do that search, the same algorithm as pathfinding, except the "nodes" being searched are world-states instead of map tiles.
Concretely: if you slam a door on a pursuing soldier, the planner has no scripted "player closed door" branch. It just re-evaluates its action space and finds that shooting through the window, or flanking through a side entrance, now satisfies its goal more cheaply than the blocked path does. That's why it reads as improvisation instead of a script — structurally, it is one.
Here's what an action library for a GOAP agent actually looks like as data, before the planner ever touches it:
{
"actions": [
{
"name": "move_to_enemy",
"preconditions": { "enemy_visible": true },
"effects": { "in_range": true },
"cost": 1
},
{
"name": "reload",
"preconditions": { "has_ammo": false },
"effects": { "has_ammo": true },
"cost": 2
},
{
"name": "attack_enemy",
"preconditions": { "in_range": true, "has_ammo": true },
"effects": { "enemy_dead": true },
"cost": 1
},
{
"name": "flank_left",
"preconditions": { "enemy_visible": true, "path_blocked": true },
"effects": { "in_range": true },
"cost": 3
}
]
}And here's a minimal but genuinely working planner that consumes that data — a backward-chaining search over world-states, same shape as A*, just over an abstract graph instead of a grid:
# goap_demo.py
def satisfies(state, preconditions):
return all(state.get(key) == value for key, value in preconditions.items())
def apply_effects(state, effects):
new_state = dict(state)
new_state.update(effects)
return new_state
def plan(start_state, goal, actions, max_depth=6):
frontier = [(start_state, [], 0)]
visited = set()
while frontier:
frontier.sort(key=lambda node: node[2])
state, path, cost = frontier.pop(0)
state_key = tuple(sorted(state.items()))
if state_key in visited:
continue
visited.add(state_key)
if satisfies(state, goal):
return path
if len(path) >= max_depth:
continue
for action in actions:
if satisfies(state, action["preconditions"]):
new_state = apply_effects(state, action["effects"])
frontier.append((new_state, path + [action["name"]], cost + action["cost"]))
return None
if __name__ == "__main__":
import json
with open("goap_actions.json") as f:
actions = json.load(f)["actions"]
start = {"enemy_visible": True, "in_range": False, "has_ammo": False, "path_blocked": False}
goal = {"enemy_dead": True}
result = plan(start, goal, actions)
print("Plan found:", result)$ python3 goap_demo.py
Plan found: ['move_to_enemy', 'reload', 'attack_enemy']Change path_blocked to True in the starting state and re-run it — the planner routes around the obstacle through flank_left automatically, without a single line of new control-flow code. That's the entire trick: intelligence-looking behavior falling out of search over a small, well-defined action space, instead of being enumerated by hand.
GOAP never fully displaced behavior trees industry-wide — it's more expensive to compute per tick, and harder to debug when a plan goes somewhere a designer didn't expect. But it's still shipping. Middle-earth: Shadow of Mordor's Nemesis system and several Tomb Raider titles use variants of it. It's the clearest counterexample to "game AI is just scripted if-statements," and also the clearest proof of why studios don't reach for planners by default: predictability is a feature, and a search algorithm that can surprise the player can also surprise the QA team debugging it at 2 a.m.
Behavior Trees and Utility Scoring
Behavior trees are the more common middle ground between rigid FSMs and expensive planners — a hierarchy of tasks and conditions evaluated top-down each tick, usually paired with utility scoring so the AI weighs several viable actions instead of just picking the first one that qualifies.
[Selector]
|
----------------------------------
| | |
[Attack?] [Retreat to cover?] [Flank?]
ammo > 0 health < 0.3 allies engaged
dist < range cover reachable flank path clear
| | |
score: 0.82 score: 0.41 score: 0.55
|
highest score wins → action executes this tickA working version of that selector, in code you can actually drop into a project:
# behavior_tree.py
class UtilityNode:
def __init__(self, name, condition_fn, score_fn):
self.name = name
self.condition_fn = condition_fn
self.score_fn = score_fn
def is_viable(self, context):
return self.condition_fn(context)
def score(self, context):
return self.score_fn(context) if self.is_viable(context) else -1
class UtilitySelector:
def __init__(self, children):
self.children = children
def tick(self, context):
viable = [c for c in self.children if c.is_viable(context)]
if not viable:
return None
best = max(viable, key=lambda c: c.score(context))
return best.name
attack = UtilityNode(
"attack",
condition_fn=lambda c: c["ammo"] > 0 and c["distance"] < c["range"],
score_fn=lambda c: c["ammo"] * 0.6 + (1 - c["distance"]) * 0.4,
)
retreat = UtilityNode(
"retreat_to_cover",
condition_fn=lambda c: c["health"] < 0.3,
score_fn=lambda c: (1 - c["health"]) * 0.8 + c["cover_nearby"] * 0.2,
)
flank = UtilityNode(
"flank",
condition_fn=lambda c: c["allies_engaged"] and c["flank_path_clear"],
score_fn=lambda c: c["allies_engaged"] * 0.5 + c["flank_path_clear"] * 0.5,
)
root = UtilitySelector([attack, retreat, flank])$ python3 -c "
from behavior_tree import root
context = {'ammo': 3, 'range': 15, 'distance': 8, 'health': 0.9,
'cover_nearby': 0.2, 'allies_engaged': 1, 'flank_path_clear': 1}
print(root.tick(context))
"
attackThis is why squads in games like The Last of Us Part II look like they're coordinating. They're not running any shared group intelligence — each NPC independently ticks its own selector against the same battlefield state, and cooperative-looking behavior falls out because the scoring functions were tuned to reward it. If two soldiers both score flank highest at the same moment, that's not communication, that's two independent functions returning the same number.
Studio-scale engines externalize this as data instead of hardcoded Python, so designers can retune AI without a recompile. A YAML version of the same tree — closer to what you'd actually find in an Unreal or Unity behavior-tree asset, serialized to a readable format — looks like this:
# squad_behavior.yaml
root:
type: selector
children:
- name: attack
condition: "ammo > 0 and distance < range"
score: "ammo * 0.6 + (1 - distance) * 0.4"
- name: retreat_to_cover
condition: "health < 0.3"
score: "(1 - health) * 0.8 + cover_nearby * 0.2"
- name: flank
condition: "allies_engaged and flank_path_clear"
score: "allies_engaged * 0.5 + flank_path_clear * 0.5"
Pathfinding: What Actually Gets You From A to B
None of the above matters if an NPC can't cross a room without walking into a wall. Pathfinding — A* search over a navigation mesh — is the system every other layer depends on, including GOAP's own planner, which is really running the same algorithm one level of abstraction up.
# astar.py
def a_star(start, goal, neighbors_fn, heuristic_fn):
open_set = {start}
came_from = {}
g_score = {start: 0}
f_score = {start: heuristic_fn(start, goal)}
while open_set:
current = min(open_set, key=lambda n: f_score.get(n, float("inf")))
if current == goal:
return reconstruct_path(came_from, current)
open_set.remove(current)
for neighbor, cost in neighbors_fn(current):
tentative_g = g_score[current] + cost
if tentative_g < g_score.get(neighbor, float("inf")):
came_from[neighbor] = current
g_score[neighbor] = tentative_g
f_score[neighbor] = tentative_g + heuristic_fn(neighbor, goal)
open_set.add(neighbor)
return None # no path exists
def reconstruct_path(came_from, current):
path = [current]
while current in came_from:
current = came_from[current]
path.append(current)
return list(reversed(path))You don't need to hand-roll this to try it — networkx ships a real A* implementation you can run against a grid graph in one line:
$ pip install networkx --break-system-packages
$ python3 -c "
import networkx as nx
G = nx.grid_2d_graph(10, 10)
path = nx.astar_path(G, (0, 0), (9, 9))
print(path)
"
[(0, 0), (1, 0), (2, 0), (3, 0), (4, 0), (5, 0), (6, 0), (7, 0), (8, 0), (9, 0), (9, 1), (9, 2), (9, 3), (9, 4), (9, 5), (9, 6), (9, 7), (9, 8), (9, 9)]In a shipped engine, none of this runs against a raw grid — it runs against a baked navigation mesh, a simplified walkable-surface graph generated from level geometry. Unity's navmesh baking, for instance, can be triggered outside the editor for CI pipelines:
$ Unity -batchmode -quit -projectPath "MyGame" \
-executeMethod NavMeshBuildScript.BakeAll \
-logFile navmesh_bake.log// NavMeshBuildScript.cs — invoked by -executeMethod above
using UnityEditor;
using UnityEngine.AI;
public static class NavMeshBuildScript
{
public static void BakeAll()
{
NavMeshBuilder.BuildNavMesh();
}
}The hard part isn't single-agent pathing, it's crowds: dozens of NPCs pathing around each other, destructible cover, and player-placed obstacles, all recomputed in real time without clumping or clipping through geometry. Get this wrong and it's the first thing players notice — and clip into a highlight reel.
Procedural Generation
A separate branch of "game AI" has nothing to do with NPC decision-making — it's about generating content. Roguelikes like Hades and open-world games like No Man's Sky build levels, loot, or entire planets algorithmically using Perlin/Simplex noise for terrain, wave function collapse for tile layouts, and constraint-solving to guarantee the result is actually playable — no unreachable rooms, no impossible difficulty spikes. This is procedural math, not machine learning, but it gets bucketed under "AI" because it produces content nobody hand-placed.
A generator's parameters are usually just a config file plus a seed, which is what makes runs reproducible for bug reports:
{
"seed": 8842,
"width": 64,
"height": 64,
"biome": "forest",
"noise": { "algorithm": "simplex", "octaves": 4, "persistence": 0.5 },
"constraints": { "min_reachable_area_pct": 0.85, "max_difficulty_spike": 0.2 }
}$ python3 gen_level.py --config level_config.json
Generated level saved to level_8842.json
Reachable area: 91.2% (min required: 85%)
Difficulty spike check: passedSame seed, same config, same level — every time. That determinism is why a player's bug report ("I fell through the floor on seed 8842") is actually actionable instead of unreproducible noise.
Tile-based generators — dungeon layouts, city blocks, puzzle rooms — more often use wave function collapse (WFC) than noise. The idea is constraint propagation: each tile position starts as a superposition of every tile type allowed to go there, and placing one tile collapses the valid options for its neighbors, which cascades outward until the whole grid is resolved or the generator hits a contradiction and backtracks.
# wfc_step.py — one collapse step, not the full algorithm
def collapse(grid, pos, tile, adjacency_rules):
grid[pos] = tile
for neighbor in get_neighbors(pos):
allowed = adjacency_rules[tile]
grid.possibilities[neighbor] &= allowed
if not grid.possibilities[neighbor]:
raise ContradictionError(f"No valid tile for {neighbor}")
return grid$ python3 wfc_generate.py --tileset dungeon_tiles.json --width 32 --height 32 --seed 8842
Collapsed 1024/1024 cells. 0 contradictions, 3 backtracks.
Saved to dungeon_8842.jsonThe "0 contradictions, 3 backtracks" line in that output is the generator quietly correcting itself — placing a tile, discovering two ticks later that it left a neighboring cell with no valid options, and unwinding back to try a different tile instead. That backtracking is most of what separates a WFC implementation that reliably finishes from one that hangs on certain seeds.
Where Machine Learning Actually Shows Up
Trained neural networks have a real but narrow footprint in shipped games, and the NPC dialogue case is worth walking through in more depth because it's the one most people can run themselves right now.
- Animation — motion matching and ML-driven blending (used in EA Sports FC) select and blend the most natural animation frame for a given movement instead of relying on hand-authored blend trees.
- Difficulty adjustment — some titles use lightweight models to track player performance and adjust spawn rates or aggression dynamically, though plenty of shipped "DDA" is really just simpler statistical tracking dressed up in the same name.
- Playtesting — studios increasingly run reinforcement-learning agents internally to stress-test levels, training an agent to break a map or find exploits faster than human QA can.
- NPC dialogue — this is the newest and most experimental. Nvidia's ACE and Ubisoft's Ghostwriter tools use LLMs to generate barks and background dialogue on the fly, still mostly confined to demos and filler lines rather than critical narrative, because keeping output on-brand and non-repetitive at scale is genuinely hard.
You can stand up a rough version of that dialogue pipeline locally in a few minutes. Pull a small model and prompt it as an NPC directly from the terminal:
$ ollama pull llama3.1:8b
$ ollama run llama3.1:8b "You are a grizzled tavern keeper NPC in a fantasy RPG. \
Greet a player who just walked in, in one sentence, in character."For anything a game actually calls at runtime, that model needs to sit behind a small service instead of a REPL. A minimal local stack — the model server plus a thin API a game client can hit — looks like this as a Compose file:
# docker-compose.yml
version: "3.9"
services:
ollama:
image: ollama/ollama
ports:
- "11434:11434"
volumes:
- ollama_data:/root/.ollama
npc-dialogue-api:
build: ./npc-api
environment:
- OLLAMA_HOST=http://ollama:11434
- MODEL=llama3.1:8b
ports:
- "8080:8080"
depends_on:
- ollama
volumes:
ollama_data:$ docker compose up -d
$ curl -s -X POST http://localhost:8080/npc/dialogue \
-H "Content-Type: application/json" \
-d '{"npc": "tavern_keeper", "context": "player_enters"}'
{"line": "Ale's fresh, coin's welcome, trouble is not — which one are you here for?"}That's structurally the same shape as Nvidia's ACE pipeline, just self-hosted and radically smaller: a game event triggers a call to an inference server, the server returns a line, the game displays it. The gap between this weekend project and a shipped AAA implementation is almost entirely about latency budgets, content moderation, and keeping a hundred NPCs from drifting off-brand over a ten-hour playthrough — not the core mechanism.
Why Studios Avoid "Real" AI for Core NPCs
This is the part the marketing never says out loud: it's about control, not capability. A behavior tree runs in microseconds and does exactly what a designer told it to. A trained model is slower, harder to debug, and can go off-distribution in ways nobody scripted — an enemy that's supposed to guard a doorway wandering off because training data never covered that exact room layout. GOAP already sits near the edge of what studios tolerate for this reason; a full neural policy controlling core combat behavior is well past it. Designers want behavior they can balance and patch on a Tuesday before a Friday ship date, not behavior they have to reverse-engineer to explain to QA.
That's also why the ML examples above cluster where they do: animation, background dialogue, offline playtesting. Every one of those is either non-critical to game logic (a bad animation blend looks odd, it doesn't break the game) or happens outside the live session entirely (an RL agent finding an exploit during QA doesn't need to run in the shipped build). The moment ML output can directly break combat balance or block player progress, studios route back to something deterministic.
Put rough per-tick budgets next to each other and the decision stops looking like taste and starts looking like arithmetic. A game running at 60 fps has roughly 16ms per frame for everything — rendering, physics, audio, and every NPC's AI combined:
# per-tick cost budget (illustrative, single agent)
finite_state_machine:
cost_us: 1-5 # microseconds
predictability: exact
behavior_tree_utility:
cost_us: 10-50
predictability: exact
goap_planner:
cost_us: 200-2000 # depends on action-space size and search depth
predictability: exact, but plan choice can surprise designers
small_local_llm_inference:
cost_ms: 50-500 # milliseconds, not microseconds — three orders of magnitude slower
predictability: probabilisticMultiply the FSM or behavior-tree cost by fifty NPCs on screen and you're still a rounding error against the frame budget. Multiply the LLM inference cost by fifty and you've blown the entire frame several times over — which is exactly why NPC dialogue systems built on language models run asynchronously, off the main game thread, and only ever produce text that gets displayed, never anything that gates player-facing game logic in real time.
We recommend reading Why Minecraft Runs on One Thread, and What Folia Breaks to continue reading our selection of content. The frame-budget math above is the same constraint that shapes Minecraft's entire server architecture, just measured in ticks instead of the 16ms-per-frame window this piece is working with.
FSM + A* in About 80 Lines
Everything above composes. Here's a minimal but complete guard AI that combines the FSM from the first section with the A* pathfinder, so an enemy in "chase" state actually paths toward the player instead of just flagging that it wants to.
$ mkdir guard-demo && cd guard-demo
$ python3 -m venv venv && source venv/bin/activate
$ pip install networkx --break-system-packages# guard_demo.py
import networkx as nx
class GuardAI:
def __init__(self, grid_size=10):
self.state = "patrol"
self.graph = nx.grid_2d_graph(grid_size, grid_size)
def update(self, can_see_player, health_pct, distance_to_player):
if health_pct < 0.2:
self.state = "flee"
elif can_see_player and distance_to_player < 3:
self.state = "attack"
elif can_see_player:
self.state = "chase"
else:
self.state = "patrol"
return self.state
def get_path(self, current_pos, player_pos, home_pos):
if self.state == "chase":
return nx.astar_path(self.graph, current_pos, player_pos)
if self.state == "flee":
return nx.astar_path(self.graph, current_pos, home_pos)
return [current_pos] # idle in place while patrolling in this demo
if __name__ == "__main__":
guard = GuardAI()
state = guard.update(can_see_player=True, health_pct=0.8, distance_to_player=6)
path = guard.get_path(current_pos=(0, 0), player_pos=(6, 6), home_pos=(0, 0))
print(f"state: {state}")
print(f"path: {path}")$ python3 guard_demo.py
state: chase
path: [(0, 0), (0, 1), (0, 2), (0, 3), (0, 4), (0, 5), (0, 6), (1, 6), (2, 6), (3, 6), (4, 6), (5, 6), (6, 6)]Swap the grid for your engine's actual navmesh query and the state machine for a proper behavior tree, and this is structurally the same enemy running in most shooters shipped in the last fifteen years.
The Takeaway
Artificial Intelligence in games is a grab-bag of decades-old computer science — state machines, planning search, procedural math — arranged to read as judgment. F.E.A.R. proved you could push that illusion further with real search instead of scripts, and two decades later most studios still choose predictability over it for anything that touches core combat. Machine learning is creeping in at the edges — animation, dialogue, offline QA — and you can now stand up a rough version of that dialogue pipeline yourself in the time it takes to pull a model. But the core loop deciding what an enemy does next is still, overwhelmingly, search and scoring at machine speed, not learning.