TensorSharp builds its GPU-platform archives on a hosted runner with no GPU
A native .NET LLM inference engine for GGUF models. TensorSharp provides a console application, a web-based chatbot interface, and Ollama/OpenAI-compatible HTTP APIs for programmatic access. It supports Windows/MacOS/iOS/Linux with full GPU capability
At a glance
- What is it?
- zhongkaifu/TensorSharp is a native .NET inference engine for GGUF models with CUDA, GGML and Apple Metal backends, a browser chat UI and OpenAI-compatible APIs. Its most candid passage is the one about the newest hardware: those archives are cross-compiled in Docker on a hosted ARM64 runner without a GPU, the only real-hardware check predates the current code, and hosted CI exercises the CPU path only.
- Who is it for?
- Use it if you want a .NET-native inference stack with real batching and prefix-shared caching on by default, and you are prepared to read the model cards before trusting a family. Do not treat the newest hardware support as verified, because the project itself says the only physical-device check predates the current code and that hosted CI rechecks only the CPU and archive paths.
- Can I use it commercially?
- Yes. BSD-3-Clause is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
- Is it still maintained?
- Yes. The repository last received commits 1 day ago.
- What is it written in?
- Mainly C#, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on November 14, 2026, and from our analysis. They are not legal advice.
Star history
- zhongkaifu/TensorSharp678 ★ · +231
| Repository | 2026-09-17 | 2026-11-14 |
|---|---|---|
| zhongkaifu/TensorSharp | 447 | 678 |
231 Stars in the last 58 days, measured from the daily GitHub snapshot. Compare with other projects
Editorial analysis
The GPU platform's archives are built where no GPU exists
The quick start has a passage that is worth reading twice, because it is the most honest thing in the repository and it is about the newest hardware.
Support for a particular accelerator is documented as a separate experimental path, with its own container build and archive instructions. Three facts are stated in sequence. The archives it produces are named for one specific GPU rather than for the architecture generally, so they are not portable to other cards of the same family. Tagged releases publish those archives, and they are built inside a container on a hosted ARM64 runner that has no GPU attached. And the existing x64 CUDA archives are explicitly not suitable for this platform.
So the last sentence of the setup is the one to hold on to: a historical check on real hardware predates the current code and does not certify it, and the hosted continuous integration rechecks only the CPU and the archive paths. That is not a failure of disclosure, it is disclosure. The project tells you the GPU path is unexercised in automation and tells you the last physical test is stale.
What it leaves you is a support matrix with one untested corner, and the documentation for it in a developer file rather than in the quick start. That is the right place for it to be, and the reason to read it before you plan around that hardware.
The release list contains two version schemes and the tags miss their own dates
Three recent releases are shown and they do not share a version scheme. The oldest is a four-part numeric version of the kind .NET assemblies use, with major, minor, build and revision components. The two newer ones are calendar strings with the year in the format position where a major version would go.
The switch happened five days apart. The numeric release is dated the twelfth of September and the first calendar release the seventeenth, so the scheme changed mid-stream rather than at a boundary. A tool that consumes these tags has to handle both.
The dates do not line up either. The calendar tag named for the first of September was actually cut on the seventeenth, sixteen days later. The one named for the twenty-ninth was cut on the thirtieth, which is close. So the tag name encodes the day the work was meant to cover, not the day the artefact appeared, which is a reasonable convention and an easy one to misread as a release date.
The consequence for a consumer is that ordering by tag name and ordering by tag date give different answers for the middle release. If you pin, pin the commit or the date, not the name.
Six prebuilt archives across three platforms, and one hardware family excluded
If you do not want to build, the releases page carries self-contained archives for the command line application and the server. There are six of them: Windows on x64 with a CPU build and a CUDA build, Linux on x64 with a CPU build and a CUDA build, and macOS on arm64.
Two things follow from that list. macOS has no CUDA variant, which is obvious, and it has no x64 variant either, so an Intel Mac is not covered by the prebuilt path even though the source build instructions explicitly tell you to choose the x64 package on macOS. The source path supports a configuration the release archives do not.
The third platform is handled separately, as described elsewhere, and the note that the x64 CUDA archives do not fit it means the word CUDA in this project covers two disjoint sets of hardware with nothing in common.
So the prebuilt matrix is six cells, and the interesting question is which cell you are in. The documentation does not put them in a table, so the fastest way to know whether you are covered is to read the archive naming in the quick start rather than to infer it from the platform list.
The benchmark is hedged on the same page that sells two books
The performance claim is one of the more carefully worded in the project. It says the engine is benchmarked against a well-known native engine on identical model files and identical hardware, that results are specific to the measured model, backend and workload, and that the detailed numbers live in a separate report. There is also a line explaining that the implementation notes and the historical benchmark claims were moved out of the readme so the page stays short.
That is the right structure for a performance claim and the right reason for moving it. Quoting a number without the model and the workload is how inference engine comparisons go wrong.
The page layout is where it gets odd. Directly above the highlights, and above everything else, sits a section promoting two commercial books that teach readers to build this kind of engine from scratch, one of them paired with the companion agent project. In the version of the page this write-up is based on, each book sits under a table cell whose link has no text, so the section renders as two rows of empty boxes above two titles. Whatever the intent, a commercial book advertisement is the largest single element on the readme of an inference engine, and it is placed above the feature list rather than beside the documentation links.
None of that makes the benchmark wrong. It just means the strongest claims on the page are the ones you have to go read elsewhere.
Three backend assemblies, and one of them only works on Apple hardware
The repository root is a directory listing of assemblies, and the three that matter are the backends.
There is a CUDA backend, a GGML backend, and an MLX backend. The last is the Apple one. That means the accelerator story is three separate builds rather than one portable layer, and the runtime offers both a managed CPU path and native accelerator paths on top of them. For a project whose pitch is that it runs everywhere, the accelerator support is genuinely per-vendor, which is the normal situation and worth knowing before you assume a single binary covers your hardware.
The mobile story rides on the same split. The companion project for phones and tablets is described as bringing the same local chat and agent experience to Apple devices through the platform's own Metal backend, not through the MLX assembly in this repository. So there are two distinct Apple code paths, one here and one there.
The rest of the assembly list is the shape of a real inference engine: a core, a runtime with a separate logging assembly, a models assembly, a chat assembly, a distributed assembly, a server and its host, the command line application, native bindings for the GGML layer, an agent host, a test matrix project and a web test project. Continuous batching and the paged prefix-shared cache are on by default, with speculative decoding, tensor parallelism and configurable security boundaries described as available on request.
Five documents exist in two languages and one plan sits at the root
Two structural habits in this repository are worth noting before you contribute.
The first is bilingual documentation as a matter of policy. Five documents each exist as an English file and a Chinese counterpart: the readme, the development guide, the features file, the usage guide and the model downloads file. That is five pairs to keep in step, and nothing in the structure enforces it. The two-language link at the top of the readme is the only navigation between them.
The second is that planning documents live at the top level rather than in the documentation directory. A tensor-parallelism plan sits beside the readme and the license, at the same level as the solution file and the build configuration. The features summary lists tensor parallelism as something available when you need it, which is a shipped capability, while the plan document at the root is about the same subject. Which one describes the current state is not something the layout answers.
There are also directories that suggest tooling rather than code: a parity directory, an engine directory, a configuration directory, an images directory, a website directory and a benchmarks directory. A parity directory next to three backend assemblies reads as an attempt to keep feature coverage honest across those backends, which is the sort of thing that is either maintained carefully or abandoned quietly.
Building from source needs the SDK, and the runtime will not do
The source path has one requirement that catches people out, and the documentation is explicit about why. A build needs the full software development kit for the target runtime version; the runtime alone cannot build the project. That is stated in its own sentence with the reason attached, which is the kind of detail that saves an afternoon.
After that, the platform instructions are unremarkable and complete. On Windows, a package manager install command is given alongside a link to the vendor's own guide:
winget install Microsoft.DotNet.SDK.10On macOS, the installer is used with an explicit instruction to choose the Arm64 package on Apple silicon or the x64 package on an Intel machine. On Linux, the vendor's distribution guide is followed to pick the right package source for the distro, with a common package name given. Then you open a new terminal and confirm the expected SDK version is listed.
The solution file itself uses the newer XML-based format rather than the classic one, which is consistent with building against the newest runtime, and there are two separate property files driving the build. The verification step is cut off in the version of the page available here, so the exact command is not visible, but the intent is clear from the sentence introducing it.
Editorial conclusion
Use it if you want a .NET-native inference stack with real batching and prefix-shared caching on by default, and you are prepared to read the model cards before trusting a family. Do not treat the newest hardware support as verified, because the project itself says the only physical-device check predates the current code and that hosted CI rechecks only the CPU and archive paths. Before you pick a build, settle three things: which backend you actually need, since the CUDA, GGML and Metal assemblies are separate and the x64 CUDA archives are stated not to fit the newest platform; whether you want prebuilt archives or the SDK, since building from source needs the full .NET 10 SDK rather than the runtime; and which version line you pin, because the release list contains two different schemes and the calendar-named tags were not cut on their name dates.
Frequently asked questions
What is TensorSharp?
A native .NET inference engine for GGUF models that ships a console application, a browser chat interface, and Ollama and OpenAI-compatible HTTP APIs. It covers autoregressive text models, text diffusion, image generation and editing, and video, with both managed CPU and native accelerator backends.
What do I need to build TensorSharp from source?
The full .NET 10 SDK rather than just the runtime, because the runtime alone cannot build the project. The README gives per-platform steps including a package-manager install on Windows and an explicit Arm64 or x64 choice on macOS, then asks you to confirm a 10.0.x SDK is listed.
Does TensorSharp support an NVIDIA DGX Spark?
Through a separate experimental path with its own container build, producing CUDA 13 Linux ARM64 archives that target one specific GPU rather than generic ARM64 hardware, and the existing x64 CUDA archives are stated not to be suitable. Tagged releases publish them from a hosted ARM64 runner that has no GPU attached.
How is TensorSharp performance measured?
Against a native engine on identical model files and identical hardware, with results scoped to the measured model, backend and workload. The README says the detailed implementation notes and historical claims were moved into linked documentation so the readme stays a starting point rather than repeating the numbers.
Which GPU backends does TensorSharp have?
Three backend assemblies: one for CUDA, one for GGML, and one for MLX, which is the Apple path. The managed runtime offers CPU and native accelerator routes on top, and the separate mobile project reaches phones and tablets through the platform's own Metal backend.
Official sources
Add this badge to your README
If you maintain this project, the badge below links readers to this analysis and shows its maintenance status from the daily GitHub snapshot. Paste the markdown into your README; add ?metric=license or ?metric=stars to the image URL for a different field.
[](https://hysenlabs.com/projects/zhongkaifu-tensorsharp)Star history chart for your README
[](https://hysenlabs.com/projects/zhongkaifu-tensorsharp)Was this useful?