tiered-compaction

English |

tiered-compaction

DSH tiered context compaction plugin — replaces a single summarization with a four-tier progressive pipeline.

DSH's default compaction, once the context reaches the threshold, makes one LLM summarization call and replaces the whole history with it. Simple, but the costs are clear: every compaction is a full LLM call; the summary is constrained by the output token cap; the middle of long content is lost first.

This plugin splits compaction into four tiers. The first three tiers consume most of the volume at zero LLM cost; only when the threshold is actually reached does it spend one budget-reserved summarization.

Four-tier pipeline

There is also a bypass: goal-switch cleanup — when a new goal is explicitly declared, the old goal's todo/goal/plan are packed into a structured capsule and process content is archived, at zero LLM cost.

Why lossy operations are safe here

Three constraints run through all tiers:

1. Every tier keeps a recovery mechanism — folding has an inverse transform record; spilling has SHA-256 and the full file; evicted items record their original location (a rerun of read/grep/glob restores them); a full archive is made before summarization.
2. Superseded references are recorded in the lineage table — whenever content is superseded, an entry "original location → new location → superseder" is registered, auditable and traceable, never silently deleted.
3. The plugin itself has a circuit breaker and an arbiter — a tier that fails consecutively is automatically degraded, skipped, and warned about without blocking the main flow; the arbiter guarantees each piece of content is handled by exactly one tier.

Installation

The plugin is pure ESM + JSDoc with no build chain, shipped straight from src/ — no compilation needed before install.

Installing from npm is recommended (clear version numbers, and update automatically upgrades to the latest stable version); use the GitHub route when npm network access is unavailable.

First, confirm which profile you use

Each DSH runtime mode corresponds to one profile name; installing into a profile makes it take effect only in that mode. Whichever way you normally start DSH, that is the name to use:

Not sure which profiles you have: look at the folder names under ~/.dsh/profiles/ (on Windows, C:\Users\<you>\.dsh\profiles\).

To install into several modes at once, run each of the commands below once per mode, replacing --profile with the corresponding name.

Desktop (most common)

# Install from npm (recommended): no tag specified means the latest stable version
dsh plugin --profile desktop add dsh-tiered-compaction

# Or install from GitHub
dsh plugin --profile desktop add "github:2849236173/tiered-compaction"

Web

dsh plugin --profile web add dsh-tiered-compaction

# Or install from GitHub
dsh plugin --profile web add "github:2849236173/tiered-compaction"

headless (command line / scripts)

dsh plugin --profile headless add dsh-tiered-compaction

# Or install from GitHub
dsh plugin --profile headless add "github:2849236173/tiered-compaction"

TUI (terminal interface)

dsh plugin --profile tui add dsh-tiered-compaction

# Or install from GitHub
dsh plugin --profile tui add "github:2849236173/tiered-compaction"

Local development (code changes take effect immediately)

To modify the plugin source locally and have changes take effect after a restart, install via a symlink — replace the path with your repository location:

# Desktop as an example; for other modes replace desktop with web / headless / tui
dsh plugin --profile desktop add "D:\你的路径\tiered-compaction"

Just restart DSH — no configuration changes needed.

The plugin's bundled patch does two things at mount time: disable the official dsh-compaction-basic (yielding the compaction service slot), and insert and enable this plugin. The archive directory defaults to ~/.dsh/tiered-archive and is created automatically if it does not exist.

Signs of a successful install — these two lines appear in the startup log:

dsh-tiered-compaction 已接管 compaction 服务位（enabled=true，archiveDir=...）
dsh-tiered-compaction 观察者已注册（工具结果 → 第 1-2 层）

Updating to a new version

After the first install, running add again does not update — add is not an upgrade command. pnpm writes the git dependency's commit hash into pnpm-lock.yaml; as long as that record exists, it considers the requirement "satisfied" and skips directly:

Already up to date

The correct upgrade command is update:

# Desktop
dsh plugin --profile desktop update dsh-tiered-compaction

# Web
dsh plugin --profile web update dsh-tiered-compaction

# headless / TUI work the same way, only replace --profile

Installed from npm: update only upgrades to latest (stable version) and will not accidentally take a beta. Installed from GitHub: update upgrades to the latest commit on main.

After upgrading you must fully exit and restart DSH, otherwise the old code still in memory keeps running.

Single-writer guard (since 0.4.0, automatic)

Since 0.4.0 this plugin ships a process-level single-writer guard: no matter how many places the plugin gets mounted in one DSH process (e.g. root plane plus a preset), only the first instance registers observers and writes archives; every later instance demotes itself to a passive self-check service (no disk writes) and logs:

dsh-tiered-compaction: single-writer guard active — ... demoted to passive self-check

Seeing that line means you double-mounted: functionality is unaffected (your sessions are still compacted by the first instance), but you should collapse the mounts as described above to remove ambiguity.

Two known boundaries:

• Duplicate copies are still protected: the guard uses a Symbol.for global seat table, so even a tarball copy and a linked checkout loading simultaneously share one registry.
• Cross-process is out of scope: running two DSH processes on one machine sharing one sessions directory is beyond this guard (that is host-level deployment territory) — do not share one session store across DSH instances.

The guard ships with a companion tripwire test (test/d24-single-writer.test.js): should a future release accidentally lose the guard, pnpm test fails loudly.

If the version did not change after update (common when the repository was renamed or the install source was switched), force a re-resolution with "uninstall then install":

dsh plugin --profile desktop remove dsh-tiered-compaction
dsh plugin --profile desktop add dsh-tiered-compaction

How to confirm the upgrade succeeded: check the version actually loaded.

# Desktop; for other modes replace desktop with web / headless / tui
$base = "$env:USERPROFILE\.dsh\profiles\desktop\node_modules"
foreach ($name in @("dsh-tiered-compaction", "@local\dsh-tiered-compaction")) {
    $f = Join-Path $base "$name\package.json"
    if (Test-Path $f) { "$name → $((Get-Content $f -Raw | ConvertFrom-Json).version)" }
}

Output like dsh-tiered-compaction → 0.3.1 means that is the currently active version.

Note: DSH does not proactively notify you of new versions. To find out whether an update exists, if you installed from npm you can run npm view dsh-tiered-compaction version and compare; you can also check the repository's CHANGELOG.

Uninstall

dsh plugin --profile desktop remove dsh-tiered-compaction

After uninstalling, the official dsh-compaction-basic is restored automatically — the patch that disabled it belongs to this plugin package and is removed along with it. A restart returns you to the official compaction; no manual config rollback needed.

(If you only want to switch back to the official compaction temporarily while keeping the plugin: swap the two switches in the patch — though in that case uninstalling is simpler.)

Configuration (all optional)

Works immediately after install; the fields below only need to be filled in when customization is required:

- id: tiered-compaction
  config:
    enabled: true                        # Default true
    archiveDir: ~/.dsh/tiered-archive    # Default value, editable
    auto: true                           # Whether to trigger automatically
    summaryTarget:                       # Model used for summarization; if empty, follows the current session
      provider: deepseek-official
      model: deepseek-chat
    modelPolicies: {}                    # Per-model overrides of thresholds and retry counts

Runtime status

No need to watch it day to day. To check whether it is working normally, search the host log for tiered-compaction:

tiered-compaction 健康 window=1000000 pressure=12.7% l12=107/31repl/0fail l3=0evict spill=29 status=ok

status=ok means normal; status=warn means the self-check found an anomaly (window reading is 0, the first two tiers have failures with zero successes, pressure ≥80% with zero action, or summaryTarget configured only halfway).

At around 82% pressure a summary compaction is triggered, and a compaction/summary record appears in the log.

Development

node --test test/*.test.js              # Full test suite
node scripts/verify-e2e.mjs <会话文件>    # End-to-end pipeline (requires a real session; verifies KV prefix equivalence)

The end-to-end script verifies "whether the summary request is a strict prefix of a normal conversation request" — if identical, it is guaranteed to hit the provider's KV cache. Session files are at ~/.dsh/sessions/<workspace>/<sessionID>/session.v4.jsonl.zstd.

Known limitations

• Host version binding: the contract layer aligns with DSH 0.2.x's CompactionEngine interface; after a host major version upgrade, the method surface must be re-verified.
• Handles plain-text tool results only: rich blocks containing images or attachments are always skipped, to avoid breaking structure.
• Pressure reading depends on the request/context event: some providers do not emit this event; when missing, pressure determination falls back to a conservative path.
• The spill preview has a fixed budget: the preview is 5120 characters (head/tail 4:1). Very large output necessarily loses information — the original is fully preserved in the archive, with its path and SHA-256 given alongside the context, so it can be read back at any time.

Documentation

License

MIT, see LICENSE.