dsh-vision-plugin

中文文档 | English

A DeepSeek Harness plugin that adds image understanding to text-only models (e.g. DeepSeek) — free of charge, powered by OpenRouter's free vision models.

• Free: uses OpenRouter's :free multimodal models — no API cost
• OpenRouter-based: only needs an OpenRouter API key

Features

• describe_image tool: reads an image file from disk and returns a text description from an OpenRouter vision model
• Automatic image handling: pasted/uploaded images are displayed as-is in the chat; if the active model cannot see images, the llm/stream hook converts them to text descriptions via OpenRouter before they reach the model (the description never enters the session or the UI); if the model supports images, they pass through untouched

flowchart TD
    A[User pastes/uploads an image] --> B[Image shown as-is in chat<br/>session keeps the image block]
    B --> C{llm/stream hook<br/>does the model support images?}
    C -->|Yes| D[Image sent to the model as-is]
    C -->|No| E[OpenRouter free vision model<br/>generates a text description]
    E --> F[Description injected into the LLM request<br/>not in session, not shown]
    D --> G[Main model answers]
    F --> G

Installation

Method 1: --patch (run from the plugin directory)

cd dsh-vision-plugin
npm install
dsh web --patch ./cordis.yml

Note: the plugin paths in cordis.yml are absolute (pointing at this machine). After cloning, edit them to your own paths.

Method 2: install as a bundle (recommended)

dsh plugin --profile web add https://github.com/Agents365-ai/dsh-vision-plugin
dsh web

After installing, no --patch is needed — just run dsh web.

Note: if installation fails with ERR_PNPM_GIT_DEP_PREPARE_NOT_ALLOWED, you pulled an old version (which had a prepare build script). Update to the latest version — the script has been removed.

Uninstall

dsh plugin --profile web remove dsh-vision-plugin

(For Method 1 --patch installs there is nothing to uninstall — just drop the --patch ./cordis.yml flag.)

Optional: also remove the deepseek-vision provider config from $DSH_HOME/settings.yaml (it becomes dead config after uninstall, but is harmless to leave).

Configure OpenRouter

Set the API key before starting (read only from the environment variable; the plugin fails with a clear error if it is missing):

export OPENROUTER_API_KEY=your_openrouter_api_key

Let DeepSeek accept pasted images

Harness behavior: pasting an image and failing to send with "the current model does not support images" is an admission check in the API proxy layer (dsh-host-apiproxy's prompt handler) — it runs before the agent loop, so the auto-vision plugin never gets a chance. The check is based on the inputModalities returned by resolveModelInfo, which depends only on the input declared in the config, not the model's real capability.

Note: the official deepseek provider is text-only — do not declare image input for it (via modelOverrides); that would mask the model's real capability, and the correct behavior is to reject images. Image understanding must go through the custom provider below.

deepseek-vision provider (auto-configured)

On first boot the plugin automatically writes the default deepseek-vision provider (pointing at OpenRouter) to $DSH_HOME/settings.yaml — no manual configuration needed. You can edit or remove it at any time:

llm-pi-ai:
  providers:
    deepseek-vision:
      apiKeyEnv: OPENROUTER_API_KEY
      api: openai-completions
      baseURL: https://openrouter.ai/api/v1
      reasoning: high
      models:
        - id: deepseek/deepseek-v4-pro
          input: [text, image]
          reasoningEfforts:
            off:
            high: high
            max: max
        - id: deepseek/deepseek-v4-flash
          input: [text, image]
          reasoningEfforts:
            off:
            high: high
            max: max

Then select the DeepSeek model under this provider in the Web UI model selector.

The auto-vision plugin already includes both alias model ids in its default force-convert list (DEFAULT_FORCE_MODELS) — no environment variable needed.

Only if you create other "declares image input but is actually text-only" aliases do you need to append:

export AUTO_VISION_FORCE_MODELS=provider/other-alias-model

(Environment models are merged with the built-in default list.)

Known limitations

• Switching models in an image session: switching to a model that does not support images (e.g. the official deepseek) is rejected by the Harness (model-unavailable) while the session contains images. This is a Harness safety feature, not a bug. Start a new session to switch.
• Image + tools in one message: tools are stripped when the current prompt contains an image (to stop weak models reflexively searching for files), so "look at the image and use a tool" needs two steps: ask about the image first, then have the model use tools.
• Vision model quality: free vision models occasionally return a safety verdict (e.g. User Safety: safe) or an overly short description; the plugin automatically retries with the next model.

❤️ Support

If this plugin helps you, consider supporting the author:

👤 Author

Agents365-ai

• GitHub: https://github.com/Agents365-ai
• Bilibili: https://space.bilibili.com/441831884

📄 License

MIT