dsh-vision-bridge

Give a text-only dsh model "eyes" — paste an image, it is recognized into text before the model sees it.

dsh-vision-bridge is a plugin for DeepSeek Harness (dsh). It lets a text-only model (e.g. deepseek-v4-flash) understand pasted images: before a message reaches the model, each image block is sent to an OpenAI-compatible vision endpoint (Zhipu free glm-4.6v-flash by default), recognized into text, and swapped in — so the model only ever sees text.

• Uses dsh's official agent/pre-step extension point. Non-intrusive; nothing in dsh's compiled artifacts is modified.
• Mounts via cordis.patch.yml, alongside plugins like dsh-vision.
• Free out-of-the-box (Zhipu free tier), with an automatic fallback chain against rate limits.

Install

# Clone into any standalone directory
git clone https://github.com/Xieweikang123/dsh-vision-bridge D:\project\dsh-vision-bridge

Let the model "accept" images

Before an image enters the agent, dsh checks the current model's inputModalities; if it does not include image, dsh rejects the message (MODEL_DOES_NOT_SUPPORT_IMAGES). Declare image input for your model in ~/.dsh/settings.yaml:

llm-pi-ai:
  providers:
    opencode-go:
      apiKeyEnv: OPENCODE_GO_API_KEY
      modelOverrides:
        deepseek-v4-flash:
          input:
            - text
            - image

This only lets dsh admit images into the agent request; turning the image into text is the plugin's job. If the gateway itself still rejects images (e.g. opencode-go returns unknown variant image_url), the plugin recognizes and replaces them before the image ever reaches the model, so the request contains no image block by then.

Mount the plugin

Add an insert entry to ~/.dsh/cordis.patch.yml (on Windows, name must be a file:// URL):

- insert:
    - id: dsh-vision-bridge
      name: 'file:///D:/project/dsh-vision-bridge/lib/index.js'

Configure the vision API key

The default uses Zhipu's free glm-4.6v-flash; you just need a key:

1. Create an API key at https://open.bigmodel.cn.
2. Put it in ~/.dsh/.env: VISION_API_KEY=<your-key>

Key resolution order: config.apiKey → $VISION_API_KEY → $DSH_VISION_API_KEY → $ZHIPUAI_API_KEY → $DASHSCOPE_API_KEY. A local Ollama endpoint needs no key.

Restart dsh, then paste an image and send it — the model will understand it.

Configuration

Backend cheat sheet

How it works

1. User pastes/upload an image → dsh persists it as an attachment; it appears as an { type: 'image', attachment } block in the message.
2. The plugin listens on agent/pre-step (dsh's official extension point, right before messages reach the model).
3. On an image block → attachments.readImage(ref) gets the bytes → a base64 data: URL.
4. It calls the vision endpoint's /chat/completions and gets back a text description.
5. It replaces the image block with the recognition text (keeping the user's own text), returning { kind: 'enter', messages }.
6. The text-only model receives pure text and understands the image.

Inspired by opencode-image-vision and dsh-vision, mapped to dsh's native extension points, with no extra runtime dependencies:

• The default Zhipu free tier glm-4.6v-flash, the fallback chain (glm-4.1v-thinking-flash → glm-4v-flash), and the API-key resolution order follow dsh-vision (MIT).
• The "recognize a pasted image into text for a text-only model" goal follows opencode-image-vision (MIT).

Identical images are cached by attachmentId to avoid re-recognition.

Known limitations

• The original image does not appear in the conversation: the model cannot ingest images, so the plugin substitutes text and only text lands in history.
• Recognition quality depends on the vision model: the free tier can be verbose on dense text (e.g. repeated terminal lines). glm-4.6v / qwen3.7-plus give better results.
• Changes to plugin code require a dsh restart to load (hot-reload primarily covers cordis.patch.yml config edits).

License

MIT