# DeepOpen: an intent-classification SDK that loads Laya checkpoints and cannot generate a word of text

> deepopen-com/deepopen answers predefined questions over text or JSON with labels, scores and probabilities, using non-autoregressive checkpoints from the upstream Laya repository. Its README is unusually frank about what its benchmark numbers do not prove.

**deepopen-com/deepopen** — DeepOpen is a multilingual, non-autoregressive System 1 decision engine designed for structured decision-making scenarios.

- Repository: https://github.com/deepopen-com/deepopen
- Website: www.deepopen.com
- Stars: 3,075 · Forks: 343
- Language: Python
- License: Apache-2.0
- Published: 2026-10-11 · Updated: 2026-10-30 · Language: en
- Canonical page: https://hysenlabs.com/projects/deepopen-com-deepopen

## It evaluates questions you define and returns constrained outputs

DeepOpen takes text or JSON and a set of predefined questions, then returns labels, scores and probabilities. The README states that it does not generate free-form text. Output types are constrained, predictions can still be wrong, and probabilities need validation on your own data.

The pyproject.toml description calls it a fast, non-autoregressive System 1 decision engine with calibrated probabilities. The README does not quantify the speed, so read the description as the design intent.

Three things ship together: a decision SDK, dataset-specific training strategies, and reproducible Banking77 and CLINC150 evaluations. The repository holds banking77/ and clinc150/ directories for the training work, plus research/, notebooks/, tests/ and a BENCHMARKS.md. The Python package is named `deepopen`, while upstream model IDs keep the name `laya` so that they match the loader configuration.

## Install from Git, because no PyPI release is promised

Python 3.10 or newer and Git are required. The README says the instructions do not depend on a PyPI release named `deepopen`, so install from the repository and check the version:

```bash
python -m pip install "git+https://github.com/deepopen-com/deepopen.git"
python -c "import deepopen; print(deepopen.__version__)"
```

For a CPU-only environment, install CPU PyTorch first:

```bash
python -m pip install torch --index-url https://download.pytorch.org/whl/cpu
python -m pip install "git+https://github.com/deepopen-com/deepopen.git"
```

GPU users should pick a PyTorch build matching their driver and CUDA environment. The benchmark training scripts have separate dependencies and their own instructions in their directories.

pyproject.toml declares torch>=2.0.0, transformers>=4.48.0, safetensors>=0.4.0, huggingface_hub>=0.20.0 and numpy>=1.20.0. Its classifiers say Development Status 4 - Beta, with Python 3.10, 3.11 and 3.12 listed. There are no GitHub releases, so the version comes from deepopen._version.

## A question has a type, instructions and criteria, and the first call downloads weights

The quick start loads the English upstream checkpoint with `deepopen.load("convaiinnovations/laya", device="cpu")`, defines a department question, and calls `agent.predict` with a body of text. The question is a dictionary with a type of choice, an instructions string and a criteria map from each option (billing, technical, sales, other) to a short description of what belongs there.

The answer carries `choice`, `probabilities` and `confidence`. The README warns that a high confidence score is not a guarantee of correctness. It also warns that the first call downloads weights and initializes the model, so it is not a warm-inference latency measurement. Use `device="cuda"` with a compatible GPU environment, and a local checkpoint directory can replace the Hub ID.

Three decision types exist. `choice` returns a label, option probabilities and confidence, as for department or intent classification. `score` returns an expected ordinal level and distribution, as for urgency. `noul` returns an estimated probability that a statement is true, as for refund request detection.

## The Router picks among three checkpoints with script heuristics

A Router chooses the checkpoint for you. `router.route(...)` shows the routing without downloading weights or running inference, and `router.predict(...)` loads the selected checkpoint on first use. Three router keys exist: `english` (ModernBERT-large, default context 512), `multilingual` (mmBERT-base, 1,024) and `typed-decisions` (ModernBERT-large, 1,024). The multilingual and typed-decisions checkpoints live in subfolders of the Laya repository on Hugging Face.

Language routing uses script and language heuristics, and the README says plainly that this is not an accuracy oracle. The typed-decisions checkpoint is not selected automatically. Select it with `model="typed-decisions"` or opt in to task detection with `auto_task_detection=True`, after validating the behavior on your workload.

Memory is the trade. The default cache keeps one model loaded. For repeated multilingual traffic, use `Router(max_loaded=2)` and `router.preload(["english", "multilingual"])`. `Router(preload=True)` loads all three models and needs memory for all of them. Preloading avoids repeated loads but does not guarantee a fixed inference latency.

## 94.1234% on BANKING77 is the best of three seeds, picked after evaluation

The headline table gives BANKING77 test accuracy of 94.1234% (2,899 of 3,080) for a single classifier with SAM seed 13, and CLINC150 test accuracy of 98.0222% (4,411 of 4,500) for a four-encoder ensemble.

The README qualifies both. The BANKING77 figure is the best test result of three SAM seeds, selected after evaluation. The three-seed mean is 94.0368%, and the README states there is no statistically established improvement over the replayed baseline. CLINC150 covers the 150 in-scope intents only, without out-of-scope detection, and combines one Laya-based specialist with three DeBERTa-v3-large models.

The screenshots do not assign an official leaderboard rank, in the README's words. An earlier table keeps an older Banking77 result of 94.0909% from a different training and execution configuration, and the README says those scores require task-specific training: installing the SDK does not reproduce them. Read these as the author's replayable evaluations, with the full protocol and prediction files on a linked Hugging Face page.

## Workflow presets are question templates, not safety guarantees

The SDK ships five template builders: `router_questions()`, `guard_questions()`, `moderation_questions()`, `triage_questions()` and `email_questions()`. The README says these are question templates and not production safety guarantees. Validate accuracy and calibrate confidence before automating consequential decisions.

That sentence limits the guard and moderation presets in particular. A guard_questions() call returns probabilities from a classifier you have not tested on your own content. The README promises no recall rate for any category.

The model weights are not in this repository. The names point at convaiinnovations/laya on Hugging Face, and pyproject.toml lists Convai Innovations as the author and a demo Space named laya-demo. A deployment therefore depends on downloading from the Hub, or on keeping a local checkpoint directory that you load by path.

## Conclusion

Use DeepOpen if you need fast routing, triage or moderation labels from text and can validate calibration on your own data before automating anything. Do not use it to produce text, to expect a leaderboard placement from the README's numbers, or to rely on language routing as an oracle. Before adopting it, run the Router on a sample of your real traffic, check the confidence values against known answers, and decide whether the Hugging Face model downloads on first use fit your deployment.

## FAQ

### What does DeepOpen do?

DeepOpen evaluates predefined questions over text or JSON and returns labels, scores and probabilities. It does not generate free-form text.

### How do I install DeepOpen?

Use Python 3.10 or newer and run python -m pip install "git+https://github.com/deepopen-com/deepopen.git". The README says installation does not depend on a PyPI release.

### Which models does DeepOpen use?

The Router chooses among english (ModernBERT-large), multilingual (mmBERT-base) and typed-decisions (ModernBERT-large) checkpoints from the upstream Laya repository.

### Are DeepOpen's benchmark scores official leaderboard results?

No. The README says its screenshots do not assign an official leaderboard rank, and the BANKING77 score is the best of three seeds selected after evaluation.

## Sources

- [deepopen-com/deepopen on GitHub](https://github.com/deepopen-com/deepopen)
- [Issues](https://github.com/deepopen-com/deepopen/issues)
- [License: Apache-2.0](https://github.com/deepopen-com/deepopen/blob/main/LICENSE)
- [README](https://github.com/deepopen-com/deepopen/blob/main/README.md)

---

Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/deepopen-com-deepopen