How AI Actually Works in Modern Video Games: FSMs to LLMs

I spent years assuming the soldiers in F.E.A.R. were running something close to real intelligence — flanking through doorways, calling out my position, regrouping when I thinned their numbers. They weren't. They were running a three-state machine bolted to a search algorithm, built by one AI programmer at Monolith in 2005. That gap — between how smart something feels and what's actually computing under the hood — is the entire story of AI in games, and two decades of "next-gen AI" marketing hasn't closed it nearly as much as you'd think. Most game AI isn't intelligent in any meaningful sense. It's rules, search, and scoring functions arranged to feel like judgment. This post walks through what's actually running — with real code you can execute, not just diagrams — from the finite state machine your first platformer boss used, up through the local LLM stack studios are experimenting with for NPC dialogue today.

Finite State Machines

The simplest and still most common form of game AI is the finite state machine. An enemy sits in one of a few discrete states — patrol, chase, attack, flee — and transitions on triggers, with no memory beyond the current state and no lookahead.

Instead of hand-rolling this, here it is against transitions, a real, maintained Python FSM library:

# guard_fsm.py
from transitions import Machine

class Guard:
    states = ["patrol", "chase", "attack", "flee"]

    def __init__(self):
        self.machine = Machine(model=self, states=Guard.states, initial="patrol")
        self.machine.add_transition("spot_player_far", "patrol", "chase")
        self.machine.add_transition("spot_player_close", ["patrol", "chase"], "attack")
        self.machine.add_transition("lose_player", ["chase", "attack"], "patrol")
        self.machine.add_transition("health_critical", "*", "flee")

guard = Guard()
print("start:", guard.state)
guard.spot_player_far()
print("after spotting player at range:", guard.state)
guard.spot_player_close()
print("after closing distance:", guard.state)
guard.health_critical()
print("after health drops below 20%:", guard.state)

$ pip install transitions --break-system-packages
$ python3 guard_fsm.py
start: patrol
after spotting player at range: chase
after closing distance: attack
after health drops below 20%: flee

That's the entire model — cheap, predictable, trivial to tune. The "*" wildcard on health_critical is transitions' real syntax for "from any state," which is exactly how a flee override is implemented in most shipped FSMs too.

We recommend reading Build Your Own Stack-Based Bytecode Virtual Machine in C to continue reading our selection of content. Same underlying idea as the FSM above.

GOAP: F.E.A.R.'s Planner

F.E.A.R. gets credited with "solving" enemy AI, and it's worth being precise about what actually changed. AI programmer Jeff Orkin didn't discard the finite state machine — F.E.A.R.'s soldiers still run one, with exactly three states: go to, animate, use smart object. What changed was how the game decides what belongs in those states: a Goal-Oriented Action Planner (GOAP), adapted from the 1971 STRIPS planning system, searches a space of actions with preconditions and effects using A* — the same algorithm as pathfinding, just over world-states instead of map tiles.

A STRIPS-style action, in the actual format real planners still use — this is genuine PDDL syntax from a working open-source solver:

And here's a minimal but genuinely working planner that consumes that data — a backward-chaining search over world-states, same shape as A*, just over an abstract graph instead of a grid:

(:action move
  :parameters (?b ?t1 ?t2)
  :precondition (and (block ?b) (on ?b ?t1) (not (on ?b ?t2)))
  :effect (and (on ?b ?t2) (not (on ?b ?t1))))

GOAP's real change, straight from Orkin's own GDC paper, is the shape of the Action class itself — no separate add/delete lists, just a flat world-state array plus two hooks for anything too expensive to represent symbolically:

class Action
{
    WORLD_STATE m_Preconditions;
    WORLD_STATE m_Effects;

    bool CheckProceduralPreconditions();
    void ActivateAction();
};

CheckProceduralPreconditions() is where a pathfinding query or line-of-sight check lives — too expensive to precompute into a symbolic world-state, so it's evaluated on demand instead. Orkin's paper lists this alongside three other concrete departures from STRIPS:

That's the actual mechanism behind an enemy "improvising" when you slam a door on it: no scripted branch, just a replanned search that finds shooting through a window now costs less than the blocked path.

Behavior Trees and Utility Scoring

Behavior trees are the more common middle ground — a hierarchy of conditions ticked top-down, usually paired with utility scoring so the AI weighs several viable actions instead of taking the first one that qualifies. Here's a squad selector against py_trees, the real behavior-tree library used across the ROS robotics ecosystem:

# squad_utility.py

import py_trees
from py_trees.common import Status


class UtilityAction:
    def __init__(
        self,
        name,
        condition_fn,
        score_fn,
        context
    ):
        self.name = name
        self.condition_fn = condition_fn
        self.score_fn = score_fn
        self.context = context

    def is_valid(self):
        return self.condition_fn(self.context)

    def score(self):
        return self.score_fn(self.context)


class UtilitySelector(
    py_trees.behaviour.Behaviour
):

    def __init__(
        self,
        name,
        actions
    ):
        super().__init__(name)

        self.actions = actions
        self.selected_action = None

    def update(self):

        valid_actions = [
            action
            for action in self.actions
            if action.is_valid()
        ]

        if not valid_actions:
            self.feedback_message = "No valid actions"
            return Status.FAILURE

        self.selected_action = max(
            valid_actions,
            key=lambda action: action.score()
        )

        score = self.selected_action.score()

        self.feedback_message = (
            f"{self.selected_action.name} "
            f"score={score:.2f}"
        )

        return Status.SUCCESS


context = {
    "ammo": 3,
    "range": 15,
    "distance": 8,
    "health": 0.9,
    "cover_nearby": 0.2,
    "allies_engaged": 1,
    "flank_path_clear": 1
}


attack = UtilityAction(

    "attack",

    lambda c:
        c["ammo"] > 0
        and c["distance"] < c["range"],

    lambda c:
        c["ammo"] * 0.6
        + (1 - c["distance"] / c["range"]) * 0.4,

    context
)


retreat = UtilityAction(

    "retreat_to_cover",

    lambda c:
        c["health"] < 0.3,

    lambda c:
        (1 - c["health"]) * 0.8
        + c["cover_nearby"] * 0.2,

    context
)


flank = UtilityAction(

    "flank",

    lambda c:
        bool(c["allies_engaged"])
        and bool(c["flank_path_clear"]),

    lambda c:
        c["allies_engaged"] * 0.5
        + c["flank_path_clear"] * 0.5,

    context
)


root = UtilitySelector(

    "squad_utility_selector",

    [
        attack,
        retreat,
        flank
    ]
)


root.tick_once()

print(
    root.feedback_message
)

pip install py_trees
python squad_utility.py

py_trees.composites.Selector runs children left to right and stops at the first SUCCESS — which is why the order children are added in matters, and why studios shipping this pattern tune priority by list order as much as by score.

Pathfinding

None of the above matters if an NPC can't cross a room. Pathfinding is A* over a navigation mesh, and you don't need to hand-roll it — networkx ships a real implementation:

# pathfinding.py

import networkx as nx


graph = nx.grid_2d_graph(
    10,
    10
)


path = nx.astar_path(
    graph,
    (0, 0),
    (9, 9)
)


print(path)
print("length:", len(path))

pip install networkx
python pathfinding.py

In a shipped engine this runs against a baked navmesh instead of a raw grid, but it's the same search — and it's the same algorithm GOAP's planner runs one level of abstraction up, over world-states instead of map coordinates. The hard part in production isn't single-agent search, it's crowds: dozens of NPCs pathing around each other and dynamic obstacles without clumping or clipping through geometry, recomputed continuously — which is why most engines layer local steering behavior on top of the raw A* result rather than re-planning every agent every frame.

Procedural Generation

Roguelikes and open-world games generate levels algorithmically instead of hand-placing them. Tile-based generation increasingly uses wave function collapse (WFC) — constraint propagation where placing one tile collapses which tiles are valid for its neighbors. The original implementation, Maxim Gumin's mxgmn/WaveFunctionCollapse, has over 25,000 GitHub stars and is the reference every other port credits. Its config format, in the real XML tile-adjacency syntax used by ports of the algorithm:

<tiles dir="./graphics/tileset/" size_x="10" size_y="10">
    <tile
        file="sky.png"
        top="sky"
        right="sky"
        bottom="sky"
        left="sky"
    />

    <tile
        file="wall_top.png"
        top="sky"
        right="wall"
        bottom="wall"
        left="wall"
    />
</tiles>

./run_wfc sample_inputs/rpg_map.xml

top/right/bottom/left define which edge strings must match between adjacent tiles — placing wall_top.png immediately rules out any neighboring tile whose adjoining edge isn't tagged wall or sky, which is the entire algorithm in miniature.

Where Machine Learning Actually Shows Up

Trained models have a real but narrow footprint: motion-matched animation blending, lightweight difficulty tracking, RL agents stress-testing levels in QA, and — newest — LLM-driven NPC dialogue. You can prompt a local model as an NPC directly from a terminal, using Ollama's real CLI:

ollama pull llama3.2
ollama run llama3.2 "You are a grizzled tavern keeper NPC. Greet a player who just walked in, in one sentence, in character."

For a game to call this at runtime it needs to sit behind a service, not a REPL — Ollama's own documented pattern for that is running it as a container:

docker run -d \
  --name ollama \
  -p 11434:11434 \
  ollama/ollama

docker exec -it ollama ollama run llama3.2

That's structurally identical to Nvidia's ACE pipeline for NPC dialogue, just self-hosted and far smaller: a game event triggers an inference call, the model returns a line, the game displays it.

Why Studios Avoid "Real" AI for Core NPCs

This is about control, not capability. A behavior tree runs in microseconds and does exactly what a designer told it to. A trained model is slower and can go off-distribution in ways nobody scripted. GOAP already sits near the edge of what studios tolerate for this reason — a full neural policy controlling core combat is well past it. Rough per-tick costs make the tradeoff look like arithmetic rather than taste: an FSM or utility-scored behavior tree costs low single-digit microseconds per agent; a GOAP search runs hundreds to low thousands of microseconds depending on action-space size; a small local LLM inference call costs tens to hundreds of milliseconds — three orders of magnitude slower. Multiply the first two by fifty on-screen NPCs and you're a rounding error against a 16ms frame budget. Multiply the LLM cost by fifty and you've blown the frame several times over, which is exactly why every ML example above is either non-critical to game logic (a bad animation blend looks odd, it doesn't break the game) or happens outside the live session entirely (an RL agent finding an exploit during QA doesn't need to run in the shipped build).

We recommend reading Why Minecraft Runs on One Thread, and What Folia Breaks to continue reading our selection of content. The frame-budget math above is the same constraint that shapes Minecraft's entire server architecture, just measured in ticks instead of the 16ms-per-frame window this piece is working with.

The Takeaway

"AI" in games is a grab-bag of decades-old computer science — state machines, planning search, procedural math — arranged to read as judgment. F.E.A.R. proved you could push that illusion further with real search instead of scripts, and two decades later most studios still choose predictability over it for anything touching core combat. Machine learning is creeping in at the edges — animation, dialogue, offline QA — but the core loop deciding what an enemy does next is still, overwhelmingly, search and scoring at machine speed, not learning.

Code above was run against transitions 0.9, py_trees 2.5.0, and networkx, current as installed from PyPI. The GOAP Action class is reproduced from Jeff Orkin's 2006 GDC paper, "Three States and a Plan: The A.I. of F.E.A.R." The WFC tile-config syntax is reproduced from a real C port of Maxim Gumin's original mxgmn/WaveFunctionCollapse.