dsh-plugin-vision-bridge

⚠️ This plugin is no longer maintained (deprecated) Recent DeepSeek Harness releases support image recognition natively (paste/upload images directly to multimodal models), so this vision bridge is no longer needed. There will be no further updates or feature requests. Please use the native harness capability; this repository and its docs are kept for reference only.

A DeepSeek Harness vision bridge: keeps the native paste/drop image experience, describes each image through an OpenAI-compatible vision API in English, and feeds DeepSeek a text-only transcript.

中文

Overview

• Composer upload button: shares the native draft-image machine used by paste/drop, including thumbnails, removal, and limit validation.
• Automatic image understanding: before the prompt reaches a text-only model, the host stores each image as a durable attachment and calls an OpenAI-compatible vision API (POST {baseUrl}/chat/completions) for an English description.
• Traceable history: each send appends a vision/image display event plus a plugin context description message; the final user message stays plain text, so DeepSeek only reads text.
• describe_image tool: the model can re-inspect uploaded images; omit imageId / imageIds to inspect the latest one. When multiple images are present the plugin context includes the attachmentId, and filename, draftId, or 1-based image number are also accepted for each entry. Pass imageIds as an array to send several images together in one vision API call so the model can compare or reason across them. An optional prompt parameter overrides the default description prompt for that call, letting the model ask the vision API for specific information (for example, “Read the error message in the image” or “What color is the button?”); when omitted, the default prompt configured in Settings is used.
• Settings page: Base URL, model, write-only API key, prompt template, timeout, and image limits; includes a “View logs” entry for troubleshooting vision API calls; bilingual UI follows Settings → General → Language live, and both themes use --dsw-alias-* tokens.

When the vision API fails, the send fails and the composer draft is restored — never a silent downgrade.

Install

Install into the web profile from GitHub (requires pnpm on PATH; otherwise use the corepack fallback below):

npx @deepseek-ai/dsh plugin --profile web add "github:c-ling/dsh-plugin-vision-bridge#v1.1.2"

Or with an existing dsh binary:

dsh plugin --profile web add "github:c-ling/dsh-plugin-vision-bridge#v1.1.2"

When pnpm is not on PATH:

cd ~/.dsh/profiles/web
corepack pnpm add "github:c-ling/dsh-plugin-vision-bridge#v1.1.2"

dsh plugin forwards its arguments to pnpm and fetches the package from this repo (pnpm 9+, git required). The warning declares no dsh.bundle — installed as a plain dependency is expected: this plugin is not a profile bundle layer; it is activated by the loader row below.

Then add a loader row to ~/.dsh/profiles/web/cordis.patch.yml:

- insert:
    - id: dsh-plugin-vision-bridge
      name: 'dsh-plugin-vision-bridge'

Restart dsh web (client-modules caches package verdicts per process; new packages require a host restart), then hard-refresh the page. An image upload button appears on the left side of the composer tool row, and a Vision Recognition page appears in Settings.

For local development, install the linked package instead:

cd ~/.dsh/profiles/web
corepack pnpm add "link:/abs/path/to/dsh-plugin-vision-bridge"

Verify

curl -s http://127.0.0.1:3080/plugins/dsh-plugin-vision-bridge/client.js | head -c 60

It should print a factory bundle starting with window.__ModuleLoader__.load({. Configure Base URL and model under Settings → Vision Recognition, then paste/upload an image and send it; the history should show an image thumbnail, an English description context bubble, and a plain-text user message, in that order.

Update

dsh plugin --profile web add "github:c-ling/dsh-plugin-vision-bridge#v1.1.2"
# or: npx @deepseek-ai/dsh plugin --profile web add "github:c-ling/dsh-plugin-vision-bridge#v1.1.2"
# or: cd ~/.dsh/profiles/web && corepack pnpm add "github:c-ling/dsh-plugin-vision-bridge#v1.1.2"

Re-running the install command with the new version pin upgrades the dependency; the loader row in cordis.patch.yml stays unchanged. Restart dsh web, then hard-refresh.

Uninstall

cd ~/.dsh/profiles/web
corepack pnpm remove dsh-plugin-vision-bridge   # or: dsh plugin --profile web remove dsh-plugin-vision-bridge

Also remove the matching insert row from cordis.patch.yml, then restart dsh web. The plugin config and vision API call logs live under $DSH_HOME/storages/dsh-plugin-vision-bridge/; images are managed by the harness attachment store. Neither is deleted automatically on uninstall — clean them up manually if desired.

Development

node --check lib/index.js lib/client.js
node --test
python3 -m json.tool package.json

• The host half is a Cordis plugin using only Node built-ins: webServer routes, tools.register, ctx.attachments, and ctx.sessions.
• The client half is a hand-written factory-CJS bundle with no build step; it registers UI through conversation.input.left, conversationEvents / conversation.chat.node, and settings.section.
• The send-path wrapper is installed in a labeled ctx.effect and restores the original conversation.sendSession when the fiber is disposed.

Known limitations

• v1 accepts PNG / JPEG / WebP / GIF only; description language is fixed to English.
• The vision API key is stored in plain text at $DSH_HOME/storages/dsh-plugin-vision-bridge/config.json (same local-settings model as the rest of the harness); the HTTP API only ever returns a redacted value.
• A real multimodal adapter / model switching is out of scope for v1.

License

MIT