dsh-llm-kilo-gateway

中文 | English

A Kilo Gateway free-model provider plugin for DeepSeek Harness.

It auto-discovers Kilo Gateway's free-model catalog, registers those models as providers you can use directly in the Harness, and adds a settings page: your own API key, the available-model list, a catalog change log, and per-model token totals.

The short version: it gets you a dozen free models at no cost, but prompts and outputs sent to these free models may be logged upstream and used for training. Do not run anything sensitive through them.

⚠️ Risks and limitations (read this first)

Every claim below cites its source. The terms-of-service quotes are Kilo's own published text; the measurements are this plugin's observations of live API responses.

1. Free-model input may be logged and used for training

This plugin measured it in the model catalog: of 401 models, 16 are free, and all 16 are flagged mayTrainOnYourPrompts: true. Kilo's own documentation says:

Auto Free may route your request to providers that log prompts and outputs and use them to improve their services. Do not submit personal or confidential data when using Auto Free.

The NVIDIA-backed free endpoints state it more strictly:

Trial use only — do not submit personal or confidential data. Your usage is logged for safety purposes and to improve NVIDIA's products and services.

2. Kilo's terms grant a broad license over submitted data

Section 7(A) of the Kilo Terms of Service (kilo.ai/terms, last updated 2026-09-30) provides that by uploading data you grant Kilo a "perpetual, irrevocable, fully-paid, royalty-free, worldwide, sublicensable, transferable" right and license, "to provide and improve the Services and Kilo's other products and services."

The same section also states:

If you elect not to grant a license to any AI Model to use your Customer Data for training purposes, you may not be able to use certain AI Models.

The opt-out offered by Section 9 covers marketing email, not AI training. Community discussion (not an official statement) suggests zero data retention (ZDR) currently appears to be limited to enterprise/team plans. This plugin neither changes these terms nor offers any way around them.

3. Free quota and rate limits

• Anonymous access: the official docs say free models work for both authenticated and anonymous users, but anonymous users are limited to 200 requests/hour/IP.
• Upstream throttling: the official docs note that "some free models may be rate-limited by the upstream provider." Measured, the free pool returns 429 Provider returned error / request limited concurrency reached — that is the upstream concurrency cap, not a configuration problem.
• Free does not mean permanently free: these models all list pricing of 0 in the catalog, but upstream can change that at any time.

4. Free models disappear

The free catalog is dynamic. Measured, 3 models carry explicit expiry dates:

The rest carry no expiry date, which does not mean they are safe to depend on. Any workflow built on a free model must be prepared for that model to vanish. The settings page keeps a change log so you can see it happen.

5. Other known boundaries of this plugin

• The free pool's stability is not something this plugin controls. Measured: upstream 400s (a model whose declared output ceiling exceeds what its upstream actually accepts), 429s (concurrency throttling), and "HTTP 200 with empty content." The plugin classifies recoverable failures correctly and retries them, but a request still fails once the retry budget is spent.
• Reasoning levels reflect only what the catalog declares. Whatever levels the catalog lists is what is shown; nothing is inferred.
• expires is displayed, not enforced. The expiry date appears in the settings page but is not used to filter out expired models.
• A port conflict disables the notify endpoint. See "Notify path." This is deliberate. Note that with two hosts running at once, the old process holds the port and the new process's endpoint stays closed.

6. No guarantee is made about upstream behaviour

This plugin is a third-party client and is not affiliated with Kilo. Upstream API fields, rate-limit policy, training terms, and even endpoint addresses can change at any time. When something breaks, check whether it is an upstream change before assuming it is a plugin defect.

Features

• 🔄 Automatic free-model discovery — reads isFree: true models from the Kilo Gateway catalog; no hand-maintained list
• 📅 Scheduled refresh — every 24 hours by default, adjustable in the settings page
• 🔔 Change notifications — a toast when models are added or removed, plus a log entry
• 🧰 Full tool calling — sends tools, parses streaming delta.tool_calls, assembles an executable tool block
• 🧠 Reasoning effort — reads each model's actually declared levels from opencode.variants
• 🖼️ Image input — driven by the catalog's architecture.input_modalities; images are sent upstream as data URLs
• 📊 Token statistics — per-model call counts and token kinds (cache reads and writes broken out)
• 🔑 Bring your own key — paste an API key in the settings page; it goes into the Harness credentials service and is never echoed back
• 📋 Available-model list and change log — the catalog's current state and its history, in the settings page
• ♻️ Upstream error retry — status codes mapped onto the Harness retry vocabulary; 429/5xx/ timeouts retry automatically

Requirements

The plugin's runtime packages (dsh-llm, cordis, schemastery, …) are declared as peerDependencies with the range *, and all of them are marked optional in peerDependenciesMeta. None of that is accidental:

• They must be peerDependencies. When the Harness decides which installed instance to redirect a plugin's bare imports to, it reads exactly the peerDependencies key names from the plugin's manifest (the host function is readPeerNames). If those names appear only under dependencies, the redirect never happens and the plugin gets its own private copy of dsh-llm — a different instance from the host's, so adapter registration does not take effect.

• The range must be *. The host compares the version ranges of @deepseek-ai/dsh and @deepseek-ai/dsh-* peers against the running dsh runtime version (not against the package's own version). Measured: pinning dsh-llm to ^0.1.5-rc.2 made the host refuse to load the plugin:

  dsh: skipping profile bundle "...": Plugin dsh-llm-kilo-gateway@1.0.0 is
  incompatible with dsh 0.2.0-rc.2: peerDependencies {"@deepseek-ai/dsh-llm":"^0.1.5-rc.2"}
  

  A pinned range makes the plugin lock itself out on the next rc bump, so * is left in place and the host's compatibility check is the gate.

• They must be marked optional. This one is for npm, not the host (the host reads only the key names and ignores meta). npm's latest tag points at an older @deepseek-ai/dsh-llm, 0.0.1-rc.1; without the optional marker npm resolves and installs a wrong-version copy into the consumer's project. Measured, with the markers added 1 package and no @deepseek-ai/* directory appeared in the consumer, while the plugin's imports still resolved to the host's copy (verified with require.resolve pointing at the same file).

Installation

# Install from npm into a profile (the web profile here)
dsh plugin --profile web add dsh-llm-kilo-gateway

Installing from source uses link: (a symlink), so source edits take effect immediately:

dsh plugin --profile web add link:/path/to/dsh-llm-kilo-gateway

Desktop: the profile is named desktop and the command is the same, but use the CLI that ships with the desktop app (the host refuses dsh --profile desktop, because that profile is managed exclusively by the Electron app).

Restart the host after installing. A Kilo Gateway group appears in the model picker, and a settings card appears under Settings → Plugins.

Settings page

Reachable from the Kilo Gateway card on the sidebar Plugins page, or from the detail page of the dsh-llm-kilo-gateway bundle card.

Editable fields

baseURL and catalogUrl are not on the settings page: editing them rewrites the entry config and reloads the fiber, which makes them shell/release-layer fields.

Your own API key

"Kilo API Key" is a write-only control: the secret literal never enters the settings document and is never returned by settings.describe, so the page only knows whether a key is configured. Writes go through the Harness credentials service at the address given by the current value of apiKeyEnv; when that field is empty it falls back to the KILO_API_KEY shown in the placeholder.

This is a hard constraint: a secret must never go into a header or a settings section — the host returns the whole section verbatim to the browser.

You can also skip the page and use an environment variable:

export KILO_API_KEY=your_key_here

Read-only panels

Three panels sit at the bottom of the page, fed by the host's loopback endpoint:

• Available models — the current free models and the capabilities the catalog claims for them. Only capabilities the catalog declares are shown; nothing is inferred or filled in.
• Catalog change log — the last 50 catalog changes, newest first.
• Token statistics — usage this process accumulated, per model. The "Total (all models)" row at the bottom is a summary row, not a model.

Configuration

Retry strategy

Retrying is performed by the Harness's dsh-llm-retry. This plugin does two things only: it classifies status codes into failure codes, and it reports maxRetries as the budget.

Which failures are retryable is a classification result, not a setting — a failure a user could retry by hand is still one the request can never survive. The adapter itself retries a connection-layer hiccup only once, and only before the stream has started (a deliberately smaller budget than the host's, so the two retry loops do not multiply).

Output budget ≠ output ceiling

A catalog entry's max_completion_tokens is the most a model can emit (a capability), not what any one request should ask for (a budget). Sending the capability as the budget gets rejected upstream. This plugin uses min(catalog ceiling, 32768), matching the built-in dsh-llm-pi-ai adapter's DEFAULT_MAX_TOKENS = 32768 convention. A user can still ask for more explicitly through the Harness's maxTokens.

Notify path

The host half runs a loopback HTTP server on the configured port; the browser half polls that same port.

Both routes listen on 127.0.0.1 only, and every response carries the CORS headers, including 404s. The page and the endpoint are not same-origin, so without Access-Control-Allow-Origin the browser blocks the entire response and collapses the status code into a bare Failed to fetch — which is what previously steered this plugin's troubleshooting in the wrong direction.

The server does not fall back to a random port when the port is taken (the client addresses the configured port, so a different port means silent failure). A failed bind logs a warning and leaves the endpoint off. Diagnosing a port conflict:

netstat -ano | grep 9876                              # who holds it
Get-CimInstance Win32_Process -Filter "Name='node.exe'" | Select ProcessId,CommandLine

Current free models (16)

The table below was generated from one real fetch and uses the same source and format as the settings page's "Available models" panel. The catalog changes at any time; the settings page is authoritative.

Notes:

• Every column is a catalog-declared value; the plugin infers nothing and fills nothing in. A missing field falls back to the conservative default (text-only, no tools).
• Max output is the model's capability ceiling, not a per-request budget — see "Output budget ≠ output ceiling."
• Reasoning levels come from the catalog's opencode.variants. Level names differ per model and are passed through verbatim rather than mapped onto a fixed vocabulary. — means the catalog declares no levels, which is not the same as "does not support reasoning."
• † openrouter/free declares no output ceiling, so the plugin falls back to 32768 and marks it.
• A Tools value of ❌ means the catalog's supported_parameters does not include tools, so the plugin does not claim tool capability.
• kilo-auto/free is an auto-router entry that dispatches requests to other free models in the catalog.

Architecture

dsh-llm-kilo-gateway/
├── package.json          # Package metadata + dsh bundle / client declarations
├── cordis.patch.yml      # Cordis composition base
├── LICENSE
├── lib/
│   ├── index.js          # Host half: adapter + catalog refresh + notify server
│   ├── client.js         # Browser half: settings page + panels + change toast
│   ├── discovery.js      # Catalog fetch and capability extraction
│   ├── wire.js           # Wire-format translation (pure functions, offline-testable)
│   ├── state.js          # State persistence
│   ├── state-util.js     # Pure functions: diffing, state building, token accumulation
│   ├── server.js         # Notify / state HTTP server
│   └── types/            # .d.ts
└── test/                 # Contract tests
    └── fixtures/         # Image fixture (the tests verify its integrity)

lib/wire.js is a separate file on purpose: the wire mapping is pure functions, and it is also where protocol bugs hide, so isolating it makes it readable and testable without a network or a running host.

Flow

1. Start — synchronously read the last persisted model snapshot to seed the list, then fetch the catalog in the background
2. Compare — diff against the previous state, find additions/removals, append to the change log
3. Register — register the free models as Harness model providers
4. Refresh — on a timer (24 hours by default)
5. Notify — when the catalog changes, tell the browser half over the loopback endpoint

The model snapshot and token statistics persist to ~/.dsh/plugins/kilo-gateway/model-state.json.

Development

npm test    # 74 tests, all offline

These tests are a specification, and every bug fix ships with a test that reproduces it: mapping tool_calls to stop, dropping tool_call_id, sending CORS on 200s only, a retry policy missing its backoff fields, treating a catalog ceiling as the output budget, moving a hook below an early return — all of them fail.

lib/client.js is hand-written: the clientBundle tsdown preset that generates it has not been published, so it is written directly in the loader's lazy-CJS factory format, which is why the tests cover parts normally guaranteed by a build.

Compatibility

License

MIT — see LICENSE.

This is a third-party client and is not affiliated with Kilo. Before using it, read Kilo's Terms of Service and the guide to using Kilo for free.