dsh-ui-external-vision

中文 | English

@dsh-external/ui-external-vision

Introduction

ui-external-vision is an external vision model plugin for the DSH Web client. When the current model does not support image input but needs to recognize an image, it calls an external VL model (default qwen3.5-omni-plus through an OpenAI-compatible API) to describe the image, supporting both local image paths and image URLs.

Installation

dsh CLI (Official Project Way)

If you have the dsh CLI installed, follow the official project tutorial to install with dsh plugin:

# Install from a local plugin directory
dsh plugin --profile web add C:/Users/<user>/.dsh/plugins/ui-external-vision

# Or install from the GitHub repository
dsh plugin --profile web add github:xiyue718/dsh-ui-external-vision

Start after installation:

dsh --profile web

View the composed configuration:

dsh --profile web --dump-config

See the project documentation for details: docs/user/develop/basic/publish.md.

Build artifacts: host lib/index.js (self-contained), client lib/client.js, package dsh-external-ui-external-vision-0.1.0.tgz.

Build once before installing: DSH_CHECKOUT=<dsh checkout> bash scripts/build.sh. Once a profile installs this package, the host resolves the plugin's runtime imports from the package's own directory only, so the host half is bundled into a self-contained module (inlining schemastery, zod, and the @deepseek-ai/dsh-* helpers); an unbundled lib/index.js leaves the row disabled with failed to import.

Usage

1. Open DSH Web.
2. Go to Settings → External Vision.
3. Fill in the model provider, Base URL, model name, and API key.
4. Click "Save".
5. Open any session.
6. Send an image URL directly (for example https://example.com/image.png). When the current model does not support image input, the plugin automatically recognizes it and injects the description for the AI.
7. When the AI calls read_image and the current model does not support image input, the plugin automatically calls the external model and returns the result to the AI.

Features

• Adds an "External Vision" settings page in Settings.
• Automatic URL recognition: when a user message contains image URLs and the current model does not support image input, the plugin downloads the URL in memory (without writing to disk), calls the external VL model, and injects the description into the request context.
• Hotlink protection compatibility: when downloading images, it automatically sends an appropriate Referer (https://www.iconfont.cn/ for iconfont/alicdn, otherwise the URL origin), reducing CDN 403 rejections.
• Visible failures: if a URL cannot be fetched or recognized, the failure reason is injected into the context instead of being silently skipped.
• Automatic local image handling: when the AI calls read_image but the current model does not support image input, the plugin calls the external VL model and returns the text description as the tool result.
• read_image interception supports file_path, url, imageUrl, and file_url parameters.
• The manual "Vision" view is no longer provided.
• The API key must be entered manually and is saved through the project credentials service, matching the Models settings page.
• Provider, Base URL, and model name are persisted through the project storage domain and synced to host memory for automatic recognition.
• Validation on save: provider must not be empty; Base URL and model name must not be empty; API key must not be empty on first save. After the API key is configured, saving again with an empty key keeps the existing key.

Host API

GET /@dsh-external/ui-external-vision/api/status

GET /@dsh-external/ui-external-vision/api/config

POST /@dsh-external/ui-external-vision/api/config
Content-Type: application/json

POST /@dsh-external/ui-external-vision/api/vision
Content-Type: application/json

/api/status returns the default Base URL and default model, never the API key. /api/vision accepts imagePath or imageUrl; when imageUrl is used, the host downloads the remote image into memory, converts it to a base64 data URL, and then sends it to the external VL model.

How It Works

The plugin consists of a host half and a client half.

The host registers two interception points: agent/pre-step and tools/execute.

• In agent/pre-step, the plugin scans user messages that are about to enter the model request and extracts image URLs. It then checks whether the current model supports image input via llm.resolveModelInfo. If the model does not support images and an API key is configured, it downloads the images into memory, calls the external VL model, and injects the description as a user message with source.kind === 'plugin'.
• In tools/execute, when the model calls read_image and the current model does not support image input, the plugin intercepts the call, reads the local image or remote URL, calls the external VL model, and returns the text description as the tool result.

The API key is read from the project credentials service and never appears in request bodies or browser storage. Remote image URLs are downloaded into host memory and converted to base64 data URLs before being sent to the external model; nothing is written to disk. The client renders the External Vision settings page and saves configuration through the storage domain and credentials service.