lucidrains/clinical-calculator-tooluse: A Research Scaffold for Wells' Criteria Tool Use

What the repository is trying to do with Wells' Criteria

The README opens with a single sentence that defines the whole project: explorations into training LLMs to use clinical calculators from patient history, using open sourced models, starting with Wells' Criteria. That sentence carries three commitments. The task is tool use rather than free-text answering, the input is patient history rather than a formatted questionnaire, and the models are open source rather than a hosted clinical API. Wells' Criteria is a scoring instrument for estimating pretest probability of deep vein thrombosis, and its inputs are discrete findings such as tenderness along a deep vein, calf swelling, pitting oedema and prior DVT. A model that reads a narrative history and then emits a score has to decide which findings are present, which are absent, and which were never mentioned. The last category is where clinical scoring normally breaks, because a missing finding and a negated finding are not the same thing. The audience is narrow: researchers and engineers who already work with transformer training loops and want a concrete, bounded task to study tool invocation. Nothing in the README addresses clinicians as end users.

The citation list is the clearest statement of method

There is no architecture diagram and no pipeline description in the README, but the bibliography is informative. Singhal et al. 2022 on clinical knowledge in large language models and Nori et al. 2023 on GPT-4 medical challenge problems frame the capability question. Schick et al. 2023 on Toolformer supplies the tool-use framing, and Peng et al. 2023 on external knowledge with automated feedback supplies the correction loop. Nov et al. 2023 and Kung et al. 2022 cover evaluation of medical advice, which is the part most projects in this space skip. Two entries point at training technique rather than background. The first is S-GRPO, described in the README as early exit via reinforcement learning in reasoning models, which suggests that a reward-based training signal is under consideration for deciding when a model has enough information to stop reasoning and call the calculator. The second is a 2026 post from tensorlabbet titled MedQASWE Post-Training, which points at post-training on medical question answering. Together these entries suggest the intended method is post-training an open model with reinforcement learning on a tool-use objective, not prompt engineering a closed model. That reading is inference from a bibliography, and the README does not state it outright.

Getting it running: what the repository does not tell you

The README contains no installation instructions, no example command, no configuration keys and no dataset description. It has a title, a one-line summary, and eight BibTeX entries. There is no pip install line, no python -m invocation, no YAML or JSON snippet, and no environment variable to set. The repository metadata reports Python as the primary language and MIT as the licence, and the default branch is main. That is the full extent of the operational information available. If you clone the repository, the first thing to inspect is the top level of main for a requirements file, a training entry point, a data directory and a licence file, because none of those are confirmed by the README. Treat any tutorial that presents a runnable command for this project as unverified unless it cites a file path that exists on the branch. The honest position is that the README is a research note with citations, and the code, if present, is undocumented at the level a new user needs.

Why patient history is the hard part of this task

Most published work on clinical calculators gives the model a structured input, which reduces the problem to arithmetic. This project takes patient history as the input, which moves the difficulty upstream into extraction. A history note may say the patient has a history of DVT from five years ago, or it may say the patient denies prior DVT, or it may omit the topic. Wells' Criteria treats prior DVT as a positive point, so a model that reads a negated mention as a positive one will inflate the score. A model that treats an omitted item as negative will deflate it. Neither error is visible in the final number, which is the central evaluation problem for this kind of tool use. Any serious attempt at this task needs per-item extraction accuracy reported alongside final score accuracy, and the README does not indicate whether that split is planned. The citation to Peng et al. on automated feedback is the only hint that error correction is part of the design.

Limits, failure modes and the wrong use case

The README describes the project as explorations, and that word should be taken literally. There are no releases. There is no homepage. There is no reported evaluation, no accuracy figure and no comparison against a rule-based implementation of the score. A rule-based implementation of Wells' Criteria is a few dozen lines of code and is auditable line by line, which makes it the correct choice for any production use. The wrong use case for this repository is clinical deployment of any kind. A second limitation is scope: the README says the work will start with Wells' Criteria, which means other calculators are aspiration, not delivered functionality. A third is model dependence. The project commits to open source models, so results will vary with the base model chosen, and the README does not name one. A fourth is that tool use for a deterministic scoring rule is a strange target in the first place, since the model could in principle be given the score directly. The research value is in the extraction and the decision of when to invoke the tool, not in the arithmetic.

Alternatives and how their approach differs

The most direct alternative is not another repository but a different design: implement Wells' Criteria as ordinary code, expose it as a function, and let a model call it with pre-extracted fields. That design removes the extraction problem from the model and puts it in a form or a parser, which is auditable and testable. The trade-off is that it cannot read free text, so it fails exactly where this project is trying to succeed. A second alternative is to follow the Toolformer line of work directly and train a general model to decide when to call external tools, then treat clinical calculators as one tool among many. That approach spreads training signal across tasks and may generalise better, but it gives you less control over the clinical specifics and no guarantee that Wells' Criteria is handled correctly. A third option is to use a hosted clinical model and prompt it, which avoids training entirely. That removes the open source model constraint the README sets, adds a data handling question, and makes the extraction behaviour harder to inspect. The difference between these options is where the extraction step lives: in the model, in a form, or in a prompt.

Maintenance cost and licence position

The repository is not archived and the last push recorded in the metadata is 2026-07-22, so it is active at the time of writing. There are no releases, which means there is no versioned artefact to pin and no changelog to read before upgrading. For a research repository of this shape, the practical maintenance cost is in tracking the base model and the training library rather than in the project itself, because the project is thin. The licence is reported as MIT, which is permissive and places few obligations on reuse beyond retaining the notice. Two caveats apply. The first is that the README does not include a licence file in its text, so confirm the LICENSE file exists on main before relying on the identifier. The second is that the bibliography includes a citation marked Creative Commons Attribution Share Alike 4.0 International, which applies to that cited work and not to this repository, but it is a reminder that derived datasets may carry their own terms. This is not legal advice; check the licence file and any dataset terms yourself.

Who should pick this up, and what to check first

This is for someone who wants a small, bounded domain in which to study how a language model decides to call a deterministic tool from messy input. Wells' Criteria is a good choice for that study because the scoring rule is public, short and unambiguous, so the only hard part is extraction. If you are that reader, clone the repository and look for three things before writing any code: a training entry point, a dataset loader, and any evaluation script. Their presence or absence tells you whether this is a runnable experiment or a set of notes. If you need a calculator that a clinician can rely on, write the score as a function with typed inputs and skip the model entirely. The repository's own framing, explorations into training LLMs to use clinical calculators, is an accurate description of what it is and what it is not.

Editorial conclusion

Adopt this only as a reading and replication target if you are working on tool-use training for clinical scoring and are comfortable that the repository is an exploration rather than a product. Do not adopt it if you need a validated calculator with a documented interface, because the README supplies no installation steps, no configuration keys and no evaluation results. Verify first whether a requirements file, training script or dataset loader exists on the main branch, and confirm the licence file matches the MIT identifier reported for the repository.

Official sources

1. Issues
2. License: MIT
3. lucidrains/clinical-calculator-tooluse on GitHub
4. README