AutoSchemaKG: schema induction for knowledge graphs without a predefined ontology
This repository contains the implementation of AutoSchemaKG, a novel framework for automatic knowledge graph construction that combines schema generation via conceptualization.
At a glance
- What is it?
- AutoSchemaKG is a Python framework from HKUST-KnowComp that extracts entity and event triples from unstructured text with an LLM, then induces a schema by concept generation. It is a research pipeline with a pip-installable atlas-rag package, and it expects you to supply the model, the prompt behaviour and the storage layer yourself.
- Who is it for?
- AutoSchemaKG is worth adopting if you already have a corpus in JSON, Markdown or PDF form, an LLM endpoint you control, and a downstream use that tolerates an induced schema rather than a fixed one, such as multi-hop QA over a Neo4j graph. It is the wrong tool if you need deterministic, auditable extraction under a fixed ontology, or if you cannot budget for the LLM calls that every extraction and concept-generation stage requires.
- Can I use it commercially?
- Yes. MIT is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
- Is it still maintained?
- Yes. The repository last received commits 139 days ago.
- What is it written in?
- Mainly Python, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on September 15, 2026, and from our analysis. They are not legal advice.
DEEP OPEN-SOURCE ANALYSIS
The ontology problem AutoSchemaKG is aimed at
Most knowledge graph pipelines start from a schema someone wrote by hand: entity types, relation types, allowed subject-object pairs. That schema is a bottleneck. It has to be maintained, it has to be extended every time a new domain appears, and anything the schema does not anticipate gets dropped at extraction time. AutoSchemaKG takes the opposite position. The README describes a two-stage approach in which triples are extracted first and the schema is induced afterwards by conceptualization, so no schema has to exist before the corpus is processed. The stated goal is to create semantic bridges between information that looks unrelated, which the project frames as enabling zero-shot inferencing across domains. The intended user is a researcher or engineer with a text corpus and a downstream retrieval or question answering task, not someone who needs a fixed ontology enforced at ingest. The repository also publishes ATLAS, a family of graphs built with the framework (ATLAS-Wiki, ATLAS-Pes2o, ATLAS-CC), which is the clearest signal of the scale the authors are targeting.
Two stages: triple extraction, then concept-based schema induction
The pipeline in the README has five documented steps, and the split between them matters. First, run_extraction() calls the LLM to pull triples covering entities and events out of the source documents. Second, convert_json_to_csv() writes those triples to CSV. Third, generate_concept_csv_temp(batch_size=64) makes a second set of LLM calls, this time to produce concepts, which is the schema induction stage. Fourth, create_concept_csv() consolidates that output into a concept CSV. Fifth, convert_to_graphml() emits a GraphML file for networkx. The data flow is therefore text in, JSON triples, CSV triples, concept CSV, GraphML out. Notice that the two LLM-heavy stages are separate and separately batched: batch_size_triple=3 for extraction and batch_size_concept=16 in the config, with generate_concept_csv_temp taking its own batch_size=64 argument. That separation is the design decision worth understanding. Extraction and conceptualization can be tuned, retried and costed independently, which is useful when one of them is the expensive half on your corpus. The README does not document the prompt templates used in either stage; it points to example/example_scripts/custom_extraction/ for custom prompts and schemas, so the extraction behaviour is configurable but not described in the main documentation.
Installing atlas-rag and running the documented pipeline
The package is installed from PyPI as atlas-rag. The plain install is pip install atlas-rag. If you need NV-embed-v2 support, the README specifies the extra form pip install atlas-rag[nvembed], which pins transformers to >=4.42.4,<=4.47.1; the release note for 24/06 mentions that the NV-embed-v2 transformers dependency was separated out, so the base install does not carry that constraint. The README example imports KnowledgeGraphExtractor from atlas_rag.kg_construction.triple_extraction, ProcessingConfig from atlas_rag.kg_construction.triple_config, and LLMGenerator from atlas_rag.llm_generator. It then constructs a client, either an OpenAI-compatible client with api_key and base_url or a local transformers text-generation pipeline, wraps it with LLMGenerator(client, model_name=model_name), and passes a ProcessingConfig with keys including model_path, data_directory, filename_pattern, batch_size_triple, batch_size_concept, output_directory, max_new_tokens, max_workers and remove_doc_spaces. Input selection is by substring: filename_pattern is matched against filenames in data_directory. The example then calls run_extraction(), convert_json_to_csv(), generate_concept_csv_temp(batch_size=64), create_concept_csv() and convert_to_graphml() in that order. Nothing in the README states how long any of this takes or what a reasonable max_workers value is for a given machine.
What the repository layout tells you about scope
The top-level layout is broader than the pip package. atlas_rag/ holds kg_construction, llm_generator, retriever, utils and vectorstore, so retrieval and vector storage ship alongside construction. The example/ directory carries notebooks for the full pipeline, for using the billion-scale ATLAS graphs, and for multi-hop QA evaluation, plus production-oriented scripts under example_scripts/ covering benchmarking, custom extraction, Neo4j hosting and parallel generation. Separate top-level directories handle evaluation: EvaluateKGC for graph quality, EvaluateFactuality for factual consistency using FELM, and EvaluateGeneralTask for MMLU. neo4j_scripts/ covers database management, and tests/ holds unit tests. That spread is a signal about what the project is: a research codebase that also ships the evaluation harnesses used in its paper, rather than a narrow library with one job. If you only want triple extraction, you are installing a package whose retriever, vectorstore and evaluation modules you will not touch. The README claims state-of-the-art performance on multiple benchmarks and strong cross-domain generalization, but it does not reproduce the benchmark table or the numbers in the text, so those claims have to be checked against the linked paper rather than the README.
Where the framework gets in your way
The most concrete limitation is that the documented output formats stop short of a running system. The pipeline ends at GraphML for networkx and CSV, and the README separately links Neo4j CSV dumps on Hugging Face. Hosting is covered by example/example_scripts/neo4j_kg/ rather than by the core package, so moving from generated CSV to a queryable database is example code you adapt, not a supported API. The second limitation is cost and variance. Both extraction and concept generation call an LLM, so the total number of calls scales with corpus size, and the schema you end up with depends on the model you choose. Swap the model and the induced concepts change, which makes schema stability across runs something you have to test rather than assume. Third, the README does not document how to evaluate an induced schema against a reference ontology, and EvaluateKGC is described only as knowledge graph quality evaluation. If your requirement is a fixed, auditable set of relation types, an induced schema is a liability, not a feature: you cannot guarantee that a given relation will appear at all. Finally, the project has no retrieved releases, so version pinning has to be done against PyPI directly, and the README's own update log shows API churn, with a codebase refactor on 05/07 and new atlas-rag documentation on 05/12.
Alternatives and the real difference in approach
The obvious comparison is a conventional information extraction stack built on a fixed ontology, where you define entity and relation types up front and train or prompt a model to fill them. The difference is not accuracy in the abstract; it is where the schema comes from. A fixed-ontology pipeline gives you reproducibility and comparability across runs, and it fails loudly when the text does not fit the schema. AutoSchemaKG gives you coverage of relations nobody anticipated, at the cost of a schema that emerges from the data and can shift with the model. The repository's own ToG addition, noted in the 24/06 update, is a reminder that graph-based retrieval methods exist alongside this one and can be evaluated against the same corpora. If your workload is multi-hop QA over a corpus you control, the notebooks in example/ are aimed directly at that, including atlas_multihopqa.ipynb. If your workload is regulated data extraction where every relation type must be justified in advance, the fixed-ontology route is the honest choice, and AutoSchemaKG's conceptualization stage is work you would have to disable or constrain through custom_extraction.
Maintenance, licence and what to pin
The licence is MIT, which permits commercial use and modification provided the copyright notice and permission notice are retained; this is a factual description of the licence text, not legal advice, and you should read the LICENSE file in the repository for the exact terms. Maintenance signals are mixed. The last push is dated 2026-04-29, the repository is not archived, and the update log shows activity through 05/12 with a batch generation and refactor entry on 05/07, but no releases were retrieved, so there is no tagged version history to reason about. That combination means upgrade cost is real: you should pin the atlas-rag version you install, keep the ProcessingConfig keys you rely on under test, and re-run the pipeline on a small sample after upgrading, because the refactor entry suggests internal module paths have moved before. The transformers constraint for the nvembed extra is the one dependency pin the README states explicitly, and it is narrow (>=4.42.4,<=4.47.1), so environments that already hold a different transformers version will conflict unless you use a separate environment.
Editorial conclusion
AutoSchemaKG is worth adopting if you already have a corpus in JSON, Markdown or PDF form, an LLM endpoint you control, and a downstream use that tolerates an induced schema rather than a fixed one, such as multi-hop QA over a Neo4j graph. It is the wrong tool if you need deterministic, auditable extraction under a fixed ontology, or if you cannot budget for the LLM calls that every extraction and concept-generation stage requires. Before committing, verify three things: that the atlas-rag version you install matches the API shown in the README, since the project refactored its codebase on 05/07 and added the atlas-rag documentation on 05/12; that your chosen model works through the LLMGenerator interface, because the README example uses a local transformers pipeline while the commented alternative is an OpenAI-compatible client; and that the output format you need is covered, because the documented pipeline ends at GraphML for networkx and CSV dumps for Neo4j, not at a live graph database.
Community notes