dsh-vidlens

中文 · English

Native video evidence tools for DeepSeek Harness. Point it at a public link — or a local file — and get something a model can actually read: timecoded lines, timecoded pictures, sound classes, comment signals. 21 tools, no cloud API, no vision model, no GPU, no telemetry.

The Python engine is a separate package: vidlens (pip install vidlens-mcp). This repository is only the DSH plugin plus its long-lived bridge.

What this plugin adds over the plain MCP shell

Install

First, get the vidlens core. Either:

pipx install "vidlens-mcp[all]"          # recommended: no checkout; commands are vidlens / vidlens-mcp
# or
git clone https://github.com/cctv2333/vidlens && cd vidlens && install.cmd

Then install this plugin (pick one):

# A. Local directory (development; point at your own clone)
plugin_manager action=install_bundle target=<your path>\dsh-vidlens

# B. Straight from GitHub (no build/prepare script, so no allowBuilds permission needed)
#    Note: this really does `git clone`, and GitHub will count it as one clone
dsh plugin --profile desktop add github:cctv2333/dsh-vidlens#<sha>

# C. npm (registry, does not touch GitHub)
dsh plugin --profile desktop add dsh-vidlens

The plugin finds the Python project itself: config.projectDir → $VIDLENS_PROJECT → <this package>/../vidlens → the current directory (only directories that actually exist; no personal paths are baked in). Python is resolved as $VIDLENS_PYTHON → that project's .venv → python. With a PyPI install, either point projectDir at the pipx venv or set VIDLENS_PROJECT — the module name does not change.

Installing the bundle inserts one Host plugin entry:

- insert:
    - id: dsh-vidlens
      name: 'dsh-vidlens'
      config: {}

Override paths per machine:

- insert:
    - id: dsh-vidlens
      name: 'dsh-vidlens'
      config:
        projectDir: D:\path\to\vidlens
        pythonPath: D:\path\to\vidlens\.venv\Scripts\python.exe
        warm: true

Tools (21)

url accepts a public link or a local path.

Basics (0 download)

Reading content

Picture (images land in the conversation)

OCR is an optional extra (RapidOCR, Apache-2.0, PP-OCRv6 models shipped inside the wheel, fully offline once installed). The first call does not download in secret: it reports the size, the target directory and how to change it; call again with install: true to proceed, and the result states where it went. Not installing it breaks nothing.

Judging / long videos / self-check

Context budget (why the default is a digest)

Measured, the old "watch a video" path cost this many characters:

The problem is not one call, it is that all of it stays in context for every later turn. In a 64k window, two or three videos plus a normal conversation hits the ceiling — the "after watching the video I'm already out of context" failure.

So the default is three tiers, upgraded on demand:

1. vidlens_brief → digest (~0.7–1.5k chars: title/duration/speech coverage/chapters/ 5 evenly sampled key lines/path to the full report);
2. vidlens_window → one segment only (~0.8k chars per minute); use it for detail, not brief;
3. vidlens_brief(level="full") → the full 6–9k report, only when the user asks for it.

The full text is always written to out/<id>/brief.md, and the digest carries the path plus what to call next. vidlens_window is capped at 80 lines and says "there are N more lines, narrow the range" instead of flooding the context.

Architecture

DSH Host (index.js)
  ├─ 21 × ctx.tools.register(...)        native tools + per-tool timeoutMs
  ├─ ctx.systemPrompt.section('vidlens') the sampling protocol (meta -> plan -> sample)
  └─ Bridge (long-lived) ──stdin/stdout JSONL──▶ python -u bridge/vidlens_bridge.py
                                                  │
                                                  ├─ import vl_mcp      (reuse the tool functions)
                                                  ├─ import vidlens_web / _extra / _sanitize
                                                  └─ out/<id>/          (cache, survives restarts)

Why not just attach the MCP server: an MCP client only has one global tool timeout, so a long ASR pass gets killed as a failure; whether images reach the model depends on the client; and "what is cached / where a phrase was said" can only be answered locally. The bridge joins the already-verified Python logic with the host's native abilities without rewriting either side.

Protocol (one JSON per line)

// boot
{"op":"ready","ok":true,"pid":1234,"project":"D:\\projects\\vidlens","python":"3.12.10","tools":["meta",...]}
// request -> response
{"id":7,"tool":"sheet","args":{"url":"...","start":"0","end":"20"}}
{"id":7,"ok":true,"text":"...","images":[{"path":"...jpg","mediaType":"image/jpeg","name":"sheet_0_20000.jpg"}]}
{"id":8,"ok":false,"error":"ValueError: ...","trace":"..."}

sys.stdout is redirected to stderr before any library import, so stray print calls inside dependencies cannot corrupt the protocol; the bridge writes frames only to the saved real stdout.

Multi-process safety: a DSH session, an MCP server and a plugin bridge are different processes, and an in-process threading.Lock cannot see across them. The core provides vidlens_web.CrossProcessLock (in-process RLock + an out/.media.lock file lock via msvcrt/flock), extending the single download mouth to the whole machine. Timeout is tunable with VIDLENS_LOCK_TIMEOUT (default 900s). Measured with two processes calling video_sheet at once: the second acquires only after the first finishes — serialized, no collision.

Verification

Offline (no network, no media)

node tools/selftest.mjs
# 130/130 checks passed (tool table + bridge handshake + library/models/seek
#                        + local-path declarations + self-heal after a forced kill)

With the real pipeline (~1MB of media)

node tools/selftest.mjs --live
# 130/130 plus 3 live checks

License

MIT. The engine is vidlens (MIT) with its own attribution file for third-party components and models.