How BibGenie Compacts Context

Long-running research conversations eventually encounter a hard constraint: a model's context window is finite.

This limitation is especially visible in Zotero. A single session may span literature searches, PDF reading, citation extraction, note organization, tool calls, and repeated corrections. As the conversation grows, sending the entire history becomes increasingly expensive and will eventually exceed the model's capacity. Simply deleting older messages is not an acceptable solution: early messages often contain the research objective, key sources, user constraints, and decisions that later work depends on.

BibGenie's Context Compaction addresses this problem. It converts earlier context into a structured research checkpoint while preserving the complete conversation history and a verbatim tail of recent messages. The model receives a smaller, information-dense working context without losing track of what has been done or what should happen next.

This article starts with how an agent request grows, then explains how BibGenie applies context compaction to research workflows, Zotero data, and branchable persistent sessions.

Why an Agent Request Keeps Growing

A chat interface looks like it is appending messages one by one, but a model does not automatically remember the previous request. Each request must carry the context needed to continue the task:

[system prompt][tools][conversation history][new user message]

When the model calls a tool, the agent executes it and sends the tool call and result back together with the earlier history. A single turn can therefore contain several model requests:

request 1:
[system][tools][user]

request 2:
[system][tools][user][assistant: tool call][tool result]

request 3:
[system][tools][user][assistant: tool call][tool result][assistant: tool call][tool result]

The context is not made up only of chat bubbles. A PDF reading may return a large passage, a Zotero search may contain many records, and repeated tool calls can expand the same turn. Eventually the next request reaches the model's context window and the provider rejects it.

There are two basic choices at that point: start a new conversation without history, or convert the existing context into a smaller representation that is still sufficient to continue working. Compaction does the latter.

Context management is not deleting the oldest messages

The most obvious approaches are to retain the latest N messages or start deleting from the beginning once a limit is reached. Neither is reliable for a research agent.

First, message count has no stable relationship with token count. A message that says “continue” may use only a handful of tokens, while a PDF extraction or tool result may contain thousands. Trimming by message count says little about the context the model must actually process.

Second, old does not mean unimportant. The user's research objective, inclusion criteria, designated papers, citation style, and key decisions often appear near the beginning of a session. Removing them can leave the agent aware of the most recent sentence but unaware of why the work began.

Finally, an agent conversation is not a plain list of text messages. A complete research workflow may also contain:

• tool calls and their results;
• Zotero item keys, attachments, and locators;
• corrections to earlier conclusions;
• failed approaches that should not be repeated;
• unfinished tasks and hypotheses awaiting verification;
• multimodal content such as images.

Compaction therefore cannot be a simple truncation operation. It must reorganize information deliberately: discard replaceable process detail while preserving the state required to continue the work.

BibGenie defines compaction as follows:

The distinction matters: the context sent with the next model request is compacted; the user's history is not.

From complete history to checkpoint plus recent tail

Suppose a session contains three conversation turns:

U1 → A1 → U2 → A2 → U3 → A3

After compaction, BibGenie does not delete or rewrite these nodes. It appends an internal checkpoint, C1, to the active session:

U1 → A1 → U2 → A2 → U3 → A3 → C1
                              ↑
                    durable session leaf

C1 stores a summary of earlier context. Its firstKeptMessageId points to the first message that should remain verbatim. If U3 is that boundary, the next model request contains:

C1.summary → U3 → A3

After the user asks another question, the context becomes:

C1.summary → U3 → A3 → U4

This creates two compatible views:

• Durable history: the database and UI retain the complete U1 → A1 → … → C1 → U4 chain.
• Model projection: the model receives only the latest checkpoint, the recent verbatim tail, and messages created after the checkpoint.

The key is not merely to shorten a string. It is to separate an auditable full history from a dense working memory for the model.

When compaction runs

BibGenie provides three entry points. All three share the same model resolution, capacity estimation, and threshold logic.

1. Manual compaction

Users can select Compact when they have completed one phase of their research and want to organize the context before moving on.

Manual compaction is not constrained by the automatic threshold. It behaves more like an explicit request to create a checkpoint now:

• Before the first checkpoint, a short session can be compacted once it contains at least two turns: the first turn is summarized and the second remains verbatim.
• If a checkpoint already exists and new messages have followed it, a new checkpoint can incorporate the previous summary even when the retained boundary does not need to move. If the boundary does move, newly excluded history is folded into the summary.
• If the session contains only one turn that cannot be compacted, or the latest checkpoint is already the branch leaf, the operation succeeds as a no-op rather than showing an unhelpful error.

This lets users create checkpoints at meaningful research milestones without turning “nothing to compact” into a failure.

2. Automatic check after a completed turn

After the assistant's final response has been persisted successfully, BibGenie checks current context usage. If the session is approaching its safe limit, it starts compaction proactively.

Moving this work out of the next send path usually means the user does not have to wait for a summary before asking the next question.

The check runs only when the session is stable. If a tool call, tool result, or user approval is still pending, compaction is skipped so that an incomplete workflow is never frozen into a checkpoint.

3. Preflight before sending

The post-turn check improves latency, but the preflight check is the final correctness boundary.

Before a new user message enters AI SDK state, BibGenie temporarily adds it to the projected context and estimates the next request. This catches cases where a user pastes a large passage, attaches an image, or switches to a model with a smaller context window.

If the request is still too large after compaction, BibGenie does not send a request that is certain to fail. The draft remains available and the user is asked to shorten it.

Decisions are based on model capacity, not message count

Models differ in context-window size and maximum output length, so BibGenie does not apply a single global threshold.

The basic relationship is:

Safe input limit = Context window - Reserved tokens

The reserved budget leaves room for the next response. It is derived from the active model's context window: large-context models can reserve a more generous output budget, while smaller models reduce it instead of inheriting an unsuitable global constant.

The recent-tail budget also scales with model capacity. Compaction therefore neither flattens all recent conversation merely to save tokens nor retains a tail that cannot fit into a smaller model.

The retained boundary is selected at a complete user-turn boundary; it never cuts through an assistant response or tool result. If a single assistant/tool turn already exceeds the recent-tail budget, BibGenie chooses the nearest user boundary after it rather than retaining the oversized turn while searching for an earlier user message.

Prefer real provider usage

A precise tokenizer for every provider would add substantial complexity, while providers do not calculate context and billing in exactly the same way. BibGenie therefore uses real usage where available and estimates only the incremental remainder:

1. Find the latest assistant message after the newest checkpoint that contains valid usage.
2. Use the provider-reported token count as the baseline.
3. Estimate messages added after that request that are not yet represented in the usage data.
4. If no reliable usage exists, estimate the complete current model projection.

The result estimates the context likely to be carried into the next provider request, not the user's cumulative token spend. Billing-oriented totalUsage is not treated as context occupancy.

Text uses a conservative character approximation. Reasoning is included when a provider may replay it. Images receive a fixed context cost during preflight so that multimodal messages are not reduced to a filename. In the current plugin, user-side file parts mainly come from pasted images and Zotero screenshots.

The estimator does not claim false precision. It answers one operational question consistently: is the next request still within a safe range?

A summary is an agent-state snapshot, not a chat recap

A conventional conversation summary describes what happened. An agent checkpoint must also contain enough state for the work to continue.

The summary is generated by a separate model request rather than being appended to the normal conversation as an afterthought. That request uses a dedicated summarization prompt, does not load the regular tools, and does not ask the model to answer the user. The history is serialized with explicit roles and boundaries, then provided as untrusted material. This keeps historical instructions from being mistaken for a new task and gives the summarization input a clear, controlled shape.

BibGenie asks the summary model to preserve a fixed set of sections:

• Research Goal: the current objective;
• User Requirements and Constraints: requirements, preferences, and limits;
• Research Progress: completed, active, and blocked work;
• Findings and Evidence: current findings and their supporting evidence;
• Key Zotero Resources: important items, attachments, and Zotero keys;
• Decisions: choices that have already been made;
• Next Steps: concrete follow-up actions;
• Critical Context: any other information required to resume correctly.

The prompt also asks the model to distinguish user statements, facts returned by tools, and model inferences. This reduces the risk that a tentative model hypothesis becomes an established fact after compaction.

Serialization for research context

Before generating a summary, BibGenie converts UIMessage data into a compact, research-oriented text representation:

• ordinary text retains its content and role label;
• reasoning is excluded from the long-term checkpoint;
• tool inputs are retained, while oversized tool results are truncated;
• tool errors and denied operations are marked explicitly;
• images and attachments retain filenames and MIME types;
• Zotero data retains the fields needed to resume work.

Different Zotero resources preserve different information. For example:

• bibliographic items: item key, title, and citation text;
• PDF pages: attachment name, current page, and total pages;
• EPUB resources: section index, href, and CFI;
• annotations: annotation key, text, comment, page, and parent item key;
• collections and tags: stable identifiers and display information.

This avoids placing complete, potentially large source objects in every summary request while preserving the Zotero locators needed for subsequent tool calls.

Preventing checkpoint accumulation

A long session may be compacted more than once. BibGenie never sends every historical checkpoint to the model; only the latest checkpoint is projected.

During a second compaction, the system combines:

previous summary + newly accumulated earlier messages

into a new summary. The projection then retains only the new checkpoint:

Summary 1 + New history → Summary 2
Summary 2 + New history → Summary 3

The context therefore does not grow linearly with the number of checkpoints.

Compaction boundaries are also monotonic: a new retained boundary cannot move behind the previous checkpoint's boundary. History that has already been replaced by a summary is never expanded back into the recent tail.

Manual and automatic compaction intentionally differ here:

• Automatic compaction runs only when the retained boundary can advance and release additional context, avoiding background summaries with no capacity benefit.
• Manual compaction may reuse the boundary when new messages have appeared after a checkpoint and produce an updated summary from the previous checkpoint. If the boundary advances, newly excluded history is incorporated as well.

This distinction keeps background maintenance focused on capacity while preserving the user's ability to mark a new research phase deliberately.

The summary output budget comes from the reserved capacity and is capped by the model's own maximum output length. It is a safety ceiling for complex research sessions; the prompt still asks for concise output.

Why compaction must understand a tree-shaped session

BibGenie sessions are not simple arrays. They are message trees that can branch.

Users can retry a response, edit the latest message, or fork from a historical assistant response. Earlier nodes may be shared by multiple sessions, while each session identifies its active branch through its own durable leaf.

Compaction must therefore neither rewrite old nodes nor copy retained messages after the checkpoint. Doing so could cause:

• compaction in one branch to affect another;
• parent links on shared nodes to change;
• a retry to inherit a checkpoint that is no longer valid;
• recent messages to be stored more than once;
• the UI history to diverge from the context seen by the model.

BibGenie uses an append-only checkpoint:

C1.parentId = A3 records when and where the checkpoint was appended in the tree. C1.firstKeptMessageId = U3 defines where the verbatim model projection resumes. These are different dimensions and cannot replace each other.

A fork created from A2 does not inherit C1. A fork created from a stable assistant message after C1 naturally includes the checkpoint in its ancestor chain. No shared node needs to be copied or modified.

Treating a checkpoint as a concurrency-safe state commit

Summary generation can take several seconds. If the branch is edited, retried, or otherwise changed during that interval, a summary generated from the old history must not be committed.

BibGenie treats compaction as a version-conditional state commit. At the beginning, it records two values:

• leafId: the current end of the durable branch;
• branchRevision: a branch version that increments only when durable content actually changes.

After the summary is generated, a dedicated transaction compares both values, confirms that the tool workflow is stable, and verifies that the retained boundary still belongs to the active branch. Only then does it append the checkpoint, move the session leaf, and increment the revision. Any mismatch means the summary is stale and must be rejected.

If the branch changes while the summary is being generated, BibGenie reloads the actual durable branch instead of forcing an obsolete checkpoint onto it. Cancellation before the transaction completes rolls the commit back. Once the transaction has succeeded, in-memory state is synchronized with the committed checkpoint so the database and UI converge on the same branch.

Coordinating with Retry, Edit, Fork, and related actions

A reliable compaction feature must preserve existing session semantics, not merely generate summaries successfully.

Retry

A checkpoint is an internal message. Retry targets the latest conversational assistant response. If a checkpoint follows the retried turn, it is invalidated with the old tail, and the new response can trigger the normal capacity checks again.

Edit

When the latest real user message is edited, the following assistant response and any checkpoint after it leave the active branch. Resubmission then passes through preflight again.

Fork

Fork targets must be stable conversational assistant messages, never internal checkpoints. Whether a new branch inherits a checkpoint is determined entirely by the target node's ancestor chain.

Protecting corrupted history

When loading a session, BibGenie checks for missing parents, parent cycles, and invalid durable messages. If the topology is corrupt, the UI displays only the safely readable portion and makes the session read-only. It never silently rewrites the original database during loading or compaction.

User experience during compaction

Context compaction is infrastructure, but it should not feel like an unexplained interface freeze.

When manual or background compaction begins, the chat displays Compacting earlier context… and offers cancellation. The editor remains available, allowing the user to prepare the next question.

Only after the user explicitly submits that question is the content frozen to wait for any running compaction. BibGenie continues automatically when compaction completes. If the operation fails or is cancelled, the message is not submitted silently and the draft remains intact.

Retry, Edit, Fork, model switching, and a second compaction are temporarily unavailable while compaction runs because they would change the branch on which the summary is based. Editing the input does not alter durable history and therefore does not need to be disabled prematurely.

From the user's perspective, the outcome is straightforward:

• long research sessions do not stop abruptly as context grows;
• there is no need to start a new chat and explain the research background again;
• switching to a smaller-context model triggers a capacity reassessment before the request;
• key sources, Zotero keys, conclusions, and unfinished tasks survive;
• recent messages remain verbatim, preserving local references and tone;
• checkpoints still belong to the correct branch after the plugin restarts;
• the complete pre-compaction history remains available for inspection.

Compaction Has a Cost: Prompt Caching Starts Over

A shorter context does not necessarily make the first request after compaction cheaper. Many model services cache unchanged request prefixes. During an ordinary append-only conversation, the system prompt, tool definitions, and most of the history remain stable, so later requests can reuse that cached prefix.

Compaction deliberately changes the model context's prefix:

Before compaction:
[system][tools][older history][recent tail]
<----------- reusable cached prefix ----------->

First request after compaction:
[system][tools][summary][recent tail][new user message]
                ^
                the token sequence changes here

The recent tail itself may be unchanged, but it now follows summary instead of older history. The previous cached prefix can no longer cover it. The first request after compaction must be recomputed; subsequent requests can establish a new cache around the compacted prefix.

This is why BibGenie does not continuously delete old tool results or rewrite the context after every turn. Compaction is an intentional cache reset and should happen when context pressure justifies it or when the user explicitly wants to organize a research phase.

The goal is not the shortest possible prompt. It is a balance among four concerns:

• whether the model can still accommodate the next input and response;
• whether the information needed to continue the task is preserved;
• whether an existing prompt cache can still be reused;
• whether latency and request cost remain reasonable.

Design trade-offs: verifiability over false precision

BibGenie's implementation draws on established agent-compaction patterns and adapts them to Zotero, multimodal input, and tree-based persistence. It deliberately avoids turning the problem into an unnecessarily elaborate system:

• no supposedly universal tokenizer that claims exactness across providers;
• no physical deletion or reparenting of pre-compaction messages;
• no duplication of the recent tail;
• no automatic compaction and replay after a provider overflow;
• no creation or mutation of checkpoints through ordinary message persistence;
• no conflation of cumulative billing tokens with context occupancy.

These choices make the important properties easier to verify: when compaction runs, which branch the summary represents, which messages the model receives, what the database commits, and where the system stops after a failure.

Compaction remains lossy by nature. Its purpose is not to make a model remember every token verbatim, but to preserve task continuity within a finite context window. The full history provides traceability, the checkpoint provides working state, and the recent tail preserves local detail. Each has a clear role.

From chatbot to long-running research partner

A conventional chat system mainly asks, “How should I answer this turn?” A research agent must also ask, “What state should I carry into the next turn?”

BibGenie's Context Compaction is more than a summarization call. It combines model-capacity preflight, research-context serialization, structured checkpoints, tree-session projection, concurrent version validation, transactional persistence, and consistent behavior across Retry, Edit, and Fork.

When research spans dozens of turns, multiple papers, and extensive tool use, remembering every token matters less than consistently knowing:

• what the user ultimately wants to accomplish;
• which claims are already supported by evidence;
• which conclusions remain hypotheses;
• what work has been completed;
• what should happen next.

That is what compaction gives BibGenie: the ability to use a finite context window for an agent session that can progress over time, recover reliably, and preserve the thread of the research.