SkillAlchemy: Turning Open-World Sources into Installable Agent Skills

The gap SkillAlchemy was built to close

Skill creation is easy when someone already knows the procedure. It is hard when nobody does. The README frames the problem directly: task descriptions omit requirements, expert-written procedures may not exist, and execution traces may be unavailable. That is the normal state of affairs for a capability your team has never packaged before. You have a brief like "review RAG systems," and the brief silently assumes you know which failure modes matter, which checks are mandatory, and which steps are conditional on the retrieval stack in front of you.

SkillAlchemy treats that as a source-grounded procedure-admission problem rather than a writing problem. The audience is engineers building agent skills for Claude Code or Codex who have access to public material (documentation, repositories, papers, issue reports) but not to a resident expert. The repository topics reinforce the scope: agent-skill, skill-acquisition, skill-construction, skill-generation, skill-pipeline. It is a production pipeline, not a prompt template.

How the four-stage pipeline turns sources into a skill package

The README describes four stages. First, discover implicit requirements that the original brief left out. Second, acquire grounded procedures from heterogeneous public sources: documentation, repositories, papers, issue reports. Third, determine procedure scope, deciding whether the evidence supports a reusable instruction, a scoped example, or exclusion. Fourth, compile an installable skill package with procedures, examples, references, and supporting resources.

The third stage is the design decision worth noticing. Most retrieval-based skill generators treat every retrieved finding as equally valid and paste it into the output. SkillAlchemy instead routes each candidate procedure into one of three buckets, and the README states the intent plainly: separate reusable instructions from context-specific examples and unsupported content "instead of treating every retrieved finding as universally valid." That admission step is what makes the output a skill rather than a reading list.

The repository also ships composable pieces. The README points to individual skills under skills/, naming Lens and LEAP as core components, and describes a fusion feature that combines existing workflows, domain knowledge, or working styles into a new capability. The technical formulation lives in TECHNICAL.md and in the arXiv paper (arXiv:2608.23417), which is the place to look for the actual scoring or filtering rules; the README does not spell them out.

Installing SkillAlchemy and generating your first skill

The README gives two installation paths. The first is to ask the agent itself, which is the least error-prone if you already have Claude Code or Codex open. The second is the command line, which is what you want in a scripted or shared environment. The package name is scoped to the GitHub repository, not to a registry namespace.

npx skills add agentsope/SkillAlchemy

Individual bundled skills install the same way, by pointing at a path inside the repository. The README shows the pattern with Lens and LEAP, and a placeholder for any other bundled skill.

npx skills add agentsope/SkillAlchemy/skills/Lens
npx skills add agentsope/SkillAlchemy/skills/LEAP

Once installed, you describe the skill you want in natural language. The README's example asks for a skill that reviews RAG systems and names public documentation and research papers as the sources.

Use SkillAlchemy to create a Skill for reviewing RAG systems.
Use public documentation and research papers as sources.

Generated packages are written to output/ in the active project, according to the README. That path is the thing to check first after a run: if output/ is empty, the pipeline did not reach the compile stage. Note that the README does not document a dry-run flag, a source allowlist, or a validation command, so there is no documented way to inspect the admitted procedures before the package is written.

What the reported numbers do and do not tell you

The README reports an evaluation on 87 tasks from SkillsBench v1.1 across four agent-model configurations. SkillAlchemy has the highest overall pass rate in 3 of 4 configurations, and a 55.8% average, which the README puts at +19.9 points over no-skill execution and +8.6 points over the strongest automated skill-creation baseline. It also reports performance comparable to human-curated skills, slightly exceeding them on average.

Read the per-configuration row before you take the average as your expectation. On Codex with DeepSeek-V4-Pro, SkillAlchemy scores 43.9 against 45.7 for human-curated skills, so it does not lead in every setting. The average is doing work that the individual cells do not always support. The comparison also depends on the benchmark: SkillsBench v1.1 tasks are the population being measured, and a skill for an internal system with no public documentation is a different problem from a skill for a documented public one. The README's own framing, open-world sources, is the boundary. If your target capability has no public evidence trail, the acquisition stage has nothing to acquire.

Where the source-grounded approach breaks down

The pipeline's strength is also its constraint. Every admitted procedure has to be justified by evidence the system can retrieve. For a capability that is genuinely novel, internal, or undocumented, there is no evidence to admit, and the output will be thin or wrong. That is not a bug in the implementation; it is the premise. A team packaging a proprietary deployment runbook should not expect SkillAlchemy to invent the runbook.

There is a second failure mode in the other direction: public sources disagree. Two repositories can document contradictory approaches to the same task, and the README's scope-determination stage has to choose between a reusable instruction and an exclusion. The README states the goal, separating supported content from unsupported content, but it does not describe the tie-breaking rule. The README also does not document rollback for a generated package, so if a compiled skill degrades agent behavior, removing it is a manual file operation you have to work out from the output/ layout yourself.

Finally, the supported agents are named as Claude Code and Codex in the README badge. A team running a different agent framework is outside the documented scope.

SkillAlchemy compared with Anthropic and OpenAI skill creators

The most direct alternatives are the skill creators the README evaluates against: Anthropic Skill-Creator and OpenAI Skill-Creator, which the reported table places at 40.6 and 42.2 average pass rate respectively, against 55.8 for SkillAlchemy. The difference in approach is where the procedure comes from. A vendor skill creator is built around the model writing a skill from the brief and the model's own knowledge, which is fast and needs no retrieval infrastructure. SkillAlchemy instead goes out to public sources and then filters what it finds through an admission decision. That extra stage is what the reported +8.6 point gap over the strongest automated baseline is attributed to, and it is also what makes the pipeline slower and dependent on source availability.

Among the other systems in the table, MUSE-Autoskill (47.2) and OpenSkill (46.0) sit between the vendor creators and SkillAlchemy. The README does not describe how those two acquire procedures, so the only comparison the README supports is the score and the position that SkillAlchemy is the one it characterizes as source-grounded with an explicit scope decision.

Maintenance, licence and the cost of upgrading

SkillAlchemy is MIT licensed, stated in both the LICENSE file and package.json. MIT is permissive: you can use, modify and redistribute it, including in commercial settings. That is a statement about the licence text, not legal advice about your situation.

The repository is not archived, and the last push was on 2026-09-02, which is recent enough that the project shows no sign of being abandoned. The single release listed is v1.0.0 (SkillAlchemy-v1.0.0-release) from 2026-06-25, so the release cadence is not fast. Note that package.json declares version "v1.0" while the release tag is v1.0.0; if you pin by version, pin by the release tag rather than the manifest string.

The upgrade cost is concentrated in the generated artifacts, not the tool. Because output/ packages are compiled from sources at generation time, a new SkillAlchemy version can change what gets admitted, and regenerating a skill may produce a different package than the one you validated. There is no documented migration path for compiled skills, so treat generated packages as build artifacts you can regenerate rather than as hand-maintained files. The README does not document a changelog process beyond the CHANGELOG.md file in the repository root.

Editorial conclusion

Adopt SkillAlchemy when the capability you want to package is unfamiliar and the brief you were handed is thin, because requirement discovery and procedure admission are exactly the steps it automates. Skip it when you already have a precise, expert-written procedure, since the pipeline adds retrieval and admission work that a hand-written skill file does not need. Before relying on it, verify two things: that your agent is Claude Code or Codex, because those are the two the README names, and that the generated package under output/ actually loads in your environment, because the README documents generation but not a validation step.

Frequently asked questions

Official sources

1. agentsope/SkillAlchemy on GitHub
2. Issues
3. License: MIT
4. README
5. Releases