TV & Compendium: How RoleCall's AI Remembers Everything

Here's a problem every AI roleplay platform hits eventually: the model forgets things.

You introduce a character named Valen in turn 3. By turn 50, the model has no idea who Valen is. You describe a magic system with specific rules. The model breaks those rules two conversations later. You establish that the tavern is called The Broken Anchor. The model calls it The Rusty Hook.

The standard fix is lorebooks — structured knowledge entries that get injected into the model's context. But lorebooks have their own problem: someone has to write and maintain them. For most users, that's too much work. The lore goes stale. New facts never get recorded. The lorebook becomes a snapshot of turn 1, useless by turn 100.

At RoleCall, we built a system called TV (TunnelVision) that solves this. It's a sidecar AI that runs alongside the narrator, autonomously managing lorebook entries in real-time. It reads what's happening in the story, decides what's worth remembering, writes it down, and retrieves the right knowledge at the right time — all without the user lifting a finger.

Then we layered Compendium on top — a typed taxonomy that turns freeform lore into structured, queryable world data.

The four phases of TV

TV doesn't run once per turn. It runs in four distinct phases, each with a different job:

Phase 1: Pre-gen retrieval

Before the narrator generates anything, TV's sidecar model receives the lorebook tree overview and recent conversation history. It selects entries across four categories:

• Active — characters, locations, and powers currently in the scene
• Anticipatory — things likely to come up based on where the story is heading
• Enrichment — world rules, magic systems, cultural context that grounds the response
• Wild card — 1-2 creative picks for flavor and continuity

The model returns specific entry names or entire branch IDs. TV resolves them from the database and injects the full content into the narrator's context. By the time the narrator starts writing, it has exactly the knowledge it needs — no more, no less.

This is context budgeting in action. Instead of dumping every lorebook entry into the prompt (which would blow the token budget), TV intelligently selects what matters for this specific turn.

Phase 2: During-gen search

Sometimes the narrator realizes mid-response that it needs information it doesn't have. TV gives it a search_lorebook tool that it can call while generating. The narrator can search by query, node ID, or specific entry IDs, and the results get injected into the ongoing generation.

This is the safety net. Phase 1 handles 90% of cases, but during-gen search catches the rest — the unexpected reference, the callback the narrator wants to honor, the detail it needs to maintain consistency.

Phase 3: Post-gen writing

After the narrator finishes, TV reviews what was just written and decides what needs to be remembered. It has a toolkit for managing the lorebook:

• remember_lore — save new facts (a character appeared, a location was described, a rule was established)
• update_lore — modify existing entries (a character's status changed, a location was destroyed)
• forget_lore — soft-delete entries that are no longer true
• create_summary — narrative recaps of what happened
• reorganize_lore — move entries between categories
• merge_split_lore — consolidate duplicates or split overgrown entries

The sidecar is aware of what Phase 1 already retrieved, so it doesn't create duplicates. It also stores write records for every change — this enables swipe rollback. If the user swipes to a different narrator response, TV can undo its writes without re-running the model.

Phase 4: Summary (historian)

On a configurable interval (every N messages), a separate "historian" pass runs. It reads message windows with overlapping lookback to catch ongoing threads, and creates arc-aware summaries — narrative recaps organized by plot thread.

The historian can also revise existing summaries, retire outdated ones, and reorganize them between arcs. This keeps the lorebook from becoming a graveyard of stale information.

How TV fits in the architecture

TV is completely independent from the DM. They solve different problems:

The DM plans where the story goes. TV manages what the story knows. The narrator receives guidance from both — the DM's directive telling it what to write, and TV's retrieved lore telling it what's true about the world.

The DM also reads a compact snapshot from the lorebook (via a compendium bridge) to inform its planning decisions. If TV recorded that two characters have a strained relationship, the DM can factor that into its arc planning. But the DM never writes to the lorebook — that's TV's job.

Compendium: typed world knowledge

Standard lorebook entries are freeform text. That's flexible but messy. You end up with entries like "Valen — tall, dark hair, member of the Shadow Guild, currently in the Northern Wastes, has a complicated relationship with Sera" — a blob of unstructured facts.

Compendium replaces freeform entries with typed entries, each with a specific schema. Instead of remember_lore, TV calls specialized tools:

Each type has structured data fields. A character entry isn't just text — it has:

{
  name: "Valen",
  description: "Former Shadow Guild operative turned reluctant ally",
  role: "Deuteragonist",
  relationships: [
    { with: "Sera", nature: "romantic_tension", status: "complicated" },
    { with: "Shadow Guild", nature: "former_member", status: "hunted" }
  ],
  current_state: {
    location: "Northern Wastes",
    status: "injured, hiding"
  }
}

This structured data enables things freeform text can't:

Smart merging. When TV updates a character, arrays like relationships deduplicate intelligently — matching by extracted keys so "Brothers in Arms" and "brothers in arms" merge rather than duplicate.

Cross-reference graphs. The Compendium API can walk all entries, parse structured fields, and build a node-and-edge graph of how everything relates. Character → Organization memberships, Location → Event associations, Plot → Threat connections.

DM snapshots. The compendium bridge extracts relationship tensions, character states, and organizational dynamics into a compact ~500-token snapshot for the DM. Structured data makes this extraction reliable — no need to parse freeform text.

Queryable state. Active plots, threats, character locations — all queryable through dedicated API endpoints rather than full-text search over blob entries.

The deduplication problem

When an AI autonomously writes lorebook entries, it will create duplicates. Aggressively. "Valen" and "Valen the Shadow" and "Valen (Northern Wastes)" are obviously the same character, but a naive system would create three entries.

TV solves this with trigram deduplication — a similarity check that runs before any new entry is created. It compares the proposed entry against existing ones using trigram overlap, catching near-duplicates that exact title matching would miss. If a match is found above the similarity threshold, TV updates the existing entry instead of creating a new one.

Combined with Compendium's smart array merging, this keeps the lorebook clean even after hundreds of autonomous writes.

Swipe rollback

RoleCall supports swiping between alternative narrator responses. This creates a problem: if TV wrote lorebook entries based on Response A, and the user swipes to Response B, those entries are now wrong.

TV handles this by storing write records as metadata on each entry — _swipe_snapshot captures the entry's state before modification, _created_by_swipe flags entries that were created (not just updated) by a specific response. When the user swipes, TV queries the database for flagged entries and replays the recorded rollback. No in-memory state required — it survives server restarts.

TV ingest: bootstrapping from history

What if you have an existing conversation with hundreds of messages but no lorebook? TV Ingest handles this — a three-phase batch pipeline:

1. Search phase — discover what entries already exist (if any)
2. Facts phase — extract character details, abilities, relationships, locations from message chunks
3. Narrative phase — create arc-aware summaries, organize into hierarchy

Each phase has configurable round budgets (coarse/moderate/detailed). The model sees its own search results, previously created entries, and reasoning throughout, maintaining continuity across chunks. By the end, you have a fully populated lorebook built from conversation history.

Activity monitoring

TV operations are tracked in a session-level activity feed — a Zustand store capped at 200 entries. Each activity records:

• Type: search, remember, update, forget, summary, reorganize, merge_split, ingest
• Phase: pre-gen, during-gen, post-gen, summary
• Model ID: which model performed the operation
• Reasoning: the sidecar model's explanation for its decision

A floating UI component shows a color-coded pulse when TV is active, with an expandable feed filtered by category. Users can see exactly what TV is doing and why — the system is autonomous but never opaque.

What we learned

Building TV taught us that knowledge management is a separate discipline from storytelling. The narrator's job is to write compelling prose. Asking it to also track every fact, name, and relationship across hundreds of turns is asking it to do two fundamentally different things.

The sidecar pattern works because it respects this separation. TV doesn't interfere with generation quality — it runs alongside it, handling the bookkeeping that would otherwise degrade the narrator's output.

Compendium taught us that structure pays for itself. Freeform text is easy to write but hard to query, merge, and maintain. Typed entries cost more upfront (10 specialized tools instead of one generic one) but make everything downstream — deduplication, cross-referencing, DM integration, dashboard views — dramatically simpler.

And the biggest lesson: users don't want to maintain lorebooks. They want to tell stories and have the world remember. TV makes that possible — a quiet, autonomous system that ensures the narrator never forgets a name, a place, or a promise.