17. History compaction
When context approaches the model limit, AVL Code automatically compacts older history into a summary, so conversations can go on indefinitely (shown as "Auto-compact" in the UI).
17.1 Triggers
- Manual: session tab right-click → Compact, or
/compact. - Automatic: the assistant triggers from its own side when its context approaches the limit; doesn't interrupt the current turn.
17.2 Visibility after compaction
Compaction is append-only: original messages are kept on disk, only flagged "hidden from current turn". You can expand the compaction summary to see the originals.
17.3 Structured info preserved
A compaction summary keeps at least:
- The latest plan and todo state
- Conclusive results of key tool calls
- Key user-confirmed decisions
- Required facts (paths, version numbers, etc.)
17.4 Multiple compactions
Long conversations can go through multiple compactions; each is additive (not a replacement). The earliest content can always be reached by drilling down.
17.4.0 Compaction itself is faster and steadier
- Incremental merge: a subsequent compaction folds only the newly added part into the existing SUMMARY instead of reloading the whole session — the wait for this step in a long session is noticeably shorter.
- Long histories are summarised in parallel: when segmented summarisation is needed, the two halves are processed concurrently rather than one after another.
- A subtask's compaction input no longer snowballs: when a subagent compacts, content already excluded from the context is stripped from the input first, so it doesn't get slower with every round.
- In-flight background subtasks are deterministically carried into the summary: compaction writes the background subtasks still running into the summary, so the assistant doesn't "forget there's work in the background" afterwards.
- On-demand tool categories don't shrink back: after a compaction, tool categories already loaded on demand are re-seeded from a snapshot and stay available instead of falling back to the minimal tool set (see §9).
- Compaction time is observable: how long this step took and how many records it absorbed are visible as their own step in the Trace panel (see §9.4.2).
17.4.1 Auto-compaction abort shows the real reason
When auto-compaction is aborted, the chat area shows the specific reason instead of a generic message — so you can tell whether it was a config issue, the context, or something else. Graceful abort / session preservation behaviour is unchanged.
17.4.2 Context-overflow root-fix
Conversations could occasionally crash with an upstream 400 and auto-compaction couldn't rescue them. The real cause is usually not under-counted tokens but a single oversized tool result — e.g. a sec.* strings extraction returning close to 300 KB and nearly filling the whole context window; two back-to-back results push the next request over the window and the upstream rejects it (these rejections don't return usage info, so reactive compaction can't see them in time).
Two-layer fix:
- Tool-result clamping: every tool result is first clamped to a safe size (about 6% of the context window, safely UTF-8-character-boundary truncated with a "continue reading" marker) — eliminates "one step blowing up the window" at the source.
- Estimation guard at send time: anchored on the previous turn's real usage + a local estimate of this turn's delta; if it predicts overflow, it proactively goes through "compact then continue" instead of firing a doomed-to-be-rejected request.
17.4.3 Reliable continuation after auto-compaction
After history compaction, the AI sometimes stopped to send only a greeting / report the current time / ask "what would you like me to do?" instead of finishing the work. This release toughens it up:
- The continuation instruction is now explicitly directive: it tells the AI to autonomously continue from the next step on incomplete / in-progress / blocked tasks in the compaction summary, not to send a greeting or report the time.
- The clock prompt isn't injected at the tail of the continuation turn, so weak models don't mistake the current time for the latest focus and ignore todos.
- If continuation still stops on a question, an auto "continue" nudge is sent (mimicking your manual "continue") to push the work forward.
- The progress bar reuses the manual
/compactone (percentage / ETA / live word count) — no more vague "processing".
17.4.4 Compaction no longer false-times-out on thinking models
Auto-compaction previously used the same fixed overall deadline for every model. Reasoning / extended-thinking models can spend a while in the thinking phase before emitting visible tokens, so they got killed mid-thought and the summary failed — which broke continuation. This release switches the wait to an idle watchdog: as long as the server keeps producing (including thinking blocks), the wait extends; only true silence triggers timeout. Thinking models can now finish their compaction summary; the conversation no longer gets cut off mid-stream.
17.4.5 Whitespace-only replies no longer falsely end the turn
Reasoning models sometimes emit chunks containing only newlines / spaces in their thinking interludes. Before, the termination check read "non-empty output = still working", so those whitespace chunks were perversely interpreted as "done" and the whole turn was cut short. The check is now "empty after trimming whitespace" — mid-thought whitespace no longer false-stops; the full reasoning stays intact.
17.4.6 A failed summary no longer pollutes the history
- A failed summary request = a failed compaction: during auto-compaction, if the summary request ends in an error (even after emitting part of its content), the compaction is treated as failed and the history is kept as-is — the error message is no longer written into the session as if it were the summary, replacing a large chunk of history; a failed manual
/compactshows "Compact session failed: …". Similar auxiliary requests such as automatic renaming and intent restatement no longer use the error message as their result when they fail either. - Waiting out a rate limit isn't a failure: when the server rate-limits, the bubble shows "速率限制中，约 X 秒后继续…" (rate limited, continuing in about X seconds…) and is marked as waiting — this notice isn't written to the session record, isn't shown as an error and doesn't count as the end of the turn; pressing Stop while waiting shows it as aborted. Context compaction in the terminal UI (TUI) no longer fails because of a rate-limit wait either.
- When the step limit is reached or a repetition halt happens, the corresponding notice is placed after that step's final reply, so the order no longer gets scrambled when you reopen the session.
17.5 KV cache prefix-cache management (perf)
Upstream model services work on a "prefix cache": system prompt → tool list → conversation history is one long string; as long as bytes at the front don't change, the cache hits; one byte at the front and the whole tail re-runs.
AVL Code addresses three "per-turn churn" sources to keep the cache warm:
| Source | Mitigation |
|---|---|
| Tool list order | Sorted by tool name to be deterministic (Go map iteration was random; the biggest unconditional win every turn) |
| Goal block + memory recall | Stripped from the prefix; reinjected at the end of the turn (invisible, not persisted) — costs only the current-turn delta, doesn't invalidate the prefix |
| Project instructions (AGENTS.md) | Deliberately kept in the prefix — stable within a session, re-reads hit the cache; only invalidates on edit (the desired behaviour) |
TUI mode is treated the same way — the CLI also moves the goal block to end-of-turn ephemeral injection, so desktop and terminal cache behaviour match.
The effect: in long conversations and multi-turn tool calls, cache-hit ratio rises noticeably — faster responses and lower token cost. This optimisation is invisible to the user — no configuration needed.
17.6 Memory palace (long-term memory)
AVL Code's built-in memory palace is the cross-session long-term memory store. At the end of a conversation, HookStop triggers a memory-candidate extraction that lands in the current palace's draft inbox (_pending/ subdirectory) for your review; once approved, the entry is filed into the relevant Room. In the slash command palette: /memorize <text> writes one directly; /memory opens the palace panel.
You can switch the whole thing off: Settings → Memory palace has an "Enable memory palace" master switch at the top (on by default). With it off the entire memory subsystem stands down — no candidate extraction, no memory injection, and "Save to memory palace" in the message menu doesn't open the drawer (it tells you where to turn it back on). Toggling takes effect with zero restart.
No spending when there's nothing to recall: when the palace holds no entries at all, the read path no longer calls the model to derive search keywords; and the first-turn injection extracts once and caches, so the same prompt isn't recomputed.
Two message-menu entries were renamed to say where things actually go: "Remember this" → "Save to memory palace", "Append to result" → "Append to result draft".
17.6.1 Smarter over time: recall weighting and negative feedback
Memories are no longer sorted purely by time or match score:
- Recall + outcome dual scoring: each memory carries
recall_count(times surfaced) andsuccess_count(times the run completed well after it was used). Weighted together, memories that get hit a lot and actually help rise to the top; ones that keep getting pulled up but don't pan out slowly sink, so they stop crowding judgement. - Rare-term weighting: at retrieval, rarer / more distinctive terms count for more — the truly relevant entry is easier to surface instead of being drowned out by generic words.
- Skip if irrelevant: memories with low relevance to the current question are no longer injected wholesale; only your pinned entries are kept — saves tokens and reduces noise.
- Lessons from the judge checker auto-distill: issues found by the self-check gate's
judgechecker (see §10.11.1) are automatically distilled into memory drafts in the review inbox — next time a similar problem comes up, the lesson is right there.
17.6.2 Conflicts go to "pending review", not hard overwrite
When a new memory contradicts an existing one (same Subject + Predicate, different Object — typical of preference / decision flips), it doesn't overwrite directly — it lands in the draft inbox tagged with ConflictHint = old_memory_id:
- The frontend
MemoryReviewPanelhighlights "conflicts with existing" on that entry. - You can supersede with one click (old entry marked expired) / keep the old / reject the new.
- Complementary to
MergeHint(triples overlap heavily with an existing entry → likely duplicate, suggest merge): merge = should combine, conflict = should supersede.
17.6.3 Memory health board
Settings → Memory palace → Health Board lays out each palace's state on one screen:
| Metric | Meaning |
|---|---|
| Total / expired / never recalled | Overview |
| Room count / per-room counts | Capacity distribution |
| Hot list (Top 10) | Most-recalled — memories that pull their weight |
| Cold list (≤ 20) | Never recalled — consider pruning |
| Low-value list (≤ 20) | Recalled enough (≥ 5) but success rate ≤ 20% — negative feedback is already demoting them; consider manual review |
| Duplicate clusters (≤ 20) | Same-room memories whose triples overlap heavily (≥ 2/3 hit) — merge suggested |
| Draft inbox pending count | pending_duplicates + pending_conflicts |
Each entry links directly to the underlying memory to pin / edit / delete; clusters can merge with one click.
17.6.4 Auto-reflect (retrospective learning, GRAI+KISS)
Turn on *Settings → Agent → Reflection → "Auto-reflect (retrospective learning)"* (off by default, since it costs an LLM call) and a retrospective runs automatically when a multi-step task wraps up: it reviews the run by the GRAI frame — Goal, Result (from the run's trace digest + self-heal stats), Analysis (5-WHY, controllable causes first), Insight — and sediments actionable lessons by the KISS taxonomy (Keep / Improve / Stop / Start) so next time a similar situation can reuse them.
- Only fires at the wrap-up of a task that did multi-step work: single-turn Q&A never triggers it; it runs async and doesn't block you.
- Lessons land in the memory palace's "pending draft inbox" (
_pending, roomscratch, kinddecision, never decays) and only enter the live memory bank for retrieval after you approve them (up to 8 per run, deduped by title). - Entirely fail-safe: memory off / no LLM / parse failure are silently skipped and never affect the task itself.
The "self-heal stats" the retrospective reads are exactly the heal history from §8.5.1 — together they form the experience-learning loop: self-heal learns from history to tune its attempts, and the retrospective sediments lessons for your review.