agent-vision-toolkit: a harness-side vision layer for text-only coding agents

The problem is the harness, not the model

A coding agent running on DeepSeek or a comparable text-only model has no image channel. Paste a screenshot into the session and nothing useful happens. Open an image through the agent's built-in file or image tools and the call is refused, because the model behind it has no vision input. The README frames this bluntly: the agent is "unable to see images, with every attempt to use an image tool blocked by the system."

The project's answer is a division of labour. Vision does not have to be a property of the model. It can be a property of the harness around it. The repository states the goal as making a tool-equipped text-model agent perform on equal or better footing than a native multimodal agent that does not use this toolkit and its methods. That is a claim about the combination of tools and method, not about any single component.

The audience is narrow and specific. You need an agent that can invoke a shell, a text-only model you are already paying for, and a workflow where screenshots, UI mockups or long page captures show up regularly. If your model already handles images natively, the premise of the project does not apply to you.

Two components: vision CLIs plus a skill, and an optional proxy

The repository ships two layers. The first is a set of vision tool CLIs and a skill that teaches the agent when to use each one. The README's own description of the boundary is that any agent able to invoke a shell can use them. That is the widest possible integration surface, and it is the layer that does not depend on any particular host's plugin API.

The second layer is the optional upgrade: a transparent local proxy plus single-file native plugins, so that images pasted into the session and the agent's built-in image tools both work without extra tool installation or additional prompting. The distinction matters for adoption. The CLI layer is portable and inspectable. The proxy layer is host-specific by construction, because it has to intercept the host's image path.

The skill is named `vision-skills`, renamed from `vision-tools` on 2026-08-18 according to the release notes in the README, so the name describes the capability rather than the underlying executables. The README lists the task families it covers: multi-image understanding, image Q&A, long-screenshot OCR, frontend UI restoration, and GUI automation.

Intent-aware viewing is the actual mechanism

The most substantive design claim in the README is about what gets sent to the vision model. Rather than producing a generic description of an image, the toolkit passes along the user's or the model's latest intent, so the vision call returns the details needed for the current turn instead of a broad, unfocused description. The README puts it as capturing "what the LLM actually cares about."

That is a real architectural choice with a real cost. A stateless describe-this-image call can be cached and reused across turns. An intent-conditioned call cannot, because the answer depends on what the agent is currently trying to do. If the agent's intent is poorly formed, the vision output inherits that weakness, and the failure is harder to diagnose than a plain "the model cannot see" error.

The skill layer is what makes the approach workable. The README describes it as teaching the agent what to inspect, which tool to choose, what sequence to follow, and how to verify the final result. That is a methodology encoded as an agent skill, not a library API. It also means the quality of the outcome depends on the agent following the skill, which is not something the repository can guarantee.

Getting it running: the skill, the CLIs, and the submodule

The README advertises a one-sentence install: ask your agent to install it, and the agent follows what the README calls the verified flow. The repository also keeps shell entry points under `bin/`, and extensions under `extensions/`, both linked from the badge row at the top of the README.

One concrete setup step is documented and easy to miss. The DSH integration, `dsh-vision-toolkit`, is tracked as a Git submodule and maintained independently at Anionex/dsh-vision-toolkit. Clone with `--recurse-submodules`, or run `git submodule update --init --recursive` in an existing checkout. Skip that and the DSH profile bundle is simply absent, with no obvious error pointing at the cause.

The DSH package is described as bringing the toolkit into DSH Web and Headless profiles as a native Profile Bundle, providing ten structured visual tools covering intent-aware image Q&A, grounding, detection, tracing, cropping, pixel diff, long-screenshot OCR, foreground extraction, dominant-color analysis, and HTML screenshots. It also adds DSH Credentials, a managed isolated runtime, previewable Artifacts, Web Settings, and agent-scoped progressive tool exposure. Those are DSH-specific features. They are not part of the portable CLI layer.

The README points to a Configuration section, but the supplied material does not include its contents, so I cannot state which keys it defines or what their defaults are. Check that section in the repository before assuming a config file is required.

Where it is the wrong tool

The proxy layer is the fragile part, and the README is honest about why it exists. Its job is to make pasted images and the host's built-in image tools work. That means intercepting a host's image handling path, and host image handling paths change. A single-file native plugin is small enough to audit and patch, which is presumably the point of shipping it that way, but it is still coupled to a specific host version.

There is a second boundary worth naming. The toolkit cannot make a host that blocks image tool calls at the system level behave differently unless you install the proxy. The README describes the blocked-call situation as the problem being solved, and the seamless integration as the optional upgrade that resolves it. Without that upgrade you are relying on the agent choosing to shell out to a CLI, which depends on the skill being loaded and followed.

Finally, cost. Every image question becomes at least one extra model call to a vision-capable endpoint. The README does not state which vision models the CLIs target, and the supplied material gives no token, latency or pricing figures. If your workflow involves many screenshots per session, that multiplication is the thing to measure before rollout, not after.

Compared with putting vision in the model

The obvious alternative is to switch to a natively multimodal model and drop the toolkit entirely. That is the right answer for many teams, and it is worth saying plainly: if your provider offers a vision-capable model at a price you accept, you get image handling without a proxy, without a skill, and without a submodule.

The difference in approach is where the vision capability lives. A multimodal model puts it inside the model weights, so every image call is one round trip and there is nothing to install. agent-vision-toolkit puts it in the harness, so the text-only model stays in place and the visual work is delegated to separate CLIs. The trade is control and portability against moving parts. You keep your existing model and its pricing, and you gain a layer you can inspect and modify, at the cost of extra calls and host-specific integration code.

The README's own framing of the bet is that a tool-equipped text-model agent can outperform a native multimodal agent that does not use this toolkit and its methods. Note the qualifier. The comparison is against a multimodal agent without the method, not against a multimodal agent with it.

Maintenance, verification claims, and the MIT licence

The project is licensed MIT, which permits commercial use, modification and redistribution provided the copyright notice and permission notice are preserved. That is the standard permissive arrangement and it is the least restrictive option for a layer you might want to fork into your own harness. This is a description of the licence text, not legal advice; read LICENSE in the repository if the terms matter to your organisation.

On maintenance, the recent history is short and active. v0.1.0 landed on 2026-08-06, v0.2.0 on 2026-08-14, and the last push to main is dated 2026-08-27. The README documents a skill rename on 2026-08-18, which means anyone who wired up the old `vision-tools` name has a rename to absorb. That is the kind of churn to expect from a project at this stage.

The verification claims are specific and worth reading literally. The README says all code has been verified in real Codex + DeepSeek sessions, and that the same pipeline has been live-verified end to end in Claude Code, Pi, Oh My Pi, and OpenCode. Those five hosts are the ones with a stated verification trail. If you run something else, you are on the CLI-plus-skill path and should treat the proxy as untested for your host. The submodule arrangement adds a second maintenance surface, since `dsh-vision-toolkit` is maintained independently and moves on its own schedule.

Editorial conclusion

Adopt it if you run a shell-capable agent on a text-only model such as DeepSeek and want image Q&A, long-screenshot OCR or UI restoration without changing the model. Do not adopt it expecting a drop-in fix for a host that blocks image tool calls at the system level, and do not treat the proxy as the only path: the CLI plus skill layer works without it. Before you commit, verify three things in your own checkout: that the Python and shell requirements in the repository are met, that the model you point the vision CLIs at can actually accept image input, and that the specific host you use appears in the list of integrations the README says has been verified end to end.

Official sources

1. Anionex/agent-vision-toolkit on GitHub
2. License: MIT
3. Project website
4. README
5. Releases