smart-contract-sanctuary: a snapshot of verified contract sources, and its expiry date

The first paragraph is a warning, and it is the whole story

The README opens with an important notice, and for a data project like this one it is the first thing a reader needs.

The dataset is no longer kept up to date. The chain-specific sub-repos stopped receiving new contracts between 2023 and mid-2024, so anything verified after that is missing here. For a larger and continuously updated dataset of verified contracts, the project points at Sourcify and its open, freely available repository.

That framing changes what the repository is. It is not a live mirror of what is deployed on Ethereum and the other chains. It is a frozen snapshot of a window in time, taken when the project was actively harvesting verified sources from a block explorer's API.

That is not a criticism. A frozen corpus has uses that a live one does not: it is stable, it is versioned, it can be diffed, and it can be quoted in a paper, which is what the citation section at the end of the README is for, with APA and BibTeX entries naming Ortner, Martin and Eskandari, Shayan.

The audience is a researcher or a security engineer who wants to search real, verified contract code offline, and a reader who should treat the data as historical rather than current.

An index repository that is really a hub of submodules

This repository does not contain the contracts. It is the index repository for the smart-contract-sanctuary, and it holds one git submodule per chain, each pointing at its own repository with its own contracts directory and its own utils directory.

The chains are ethereum, arbitrum, avalanche, bsc, celo, fantom, optimism, polygon and tron, and the topics on the repository list arbitrum, avalanche-c-chain, binance-smart-chain, ethereum, maticnetwork, polygon, solidity and tron-protocol, which matches the submodules rather than the code, since there is very little code here at all.

The layout table gives the three kinds of directory. _docs holds autogenerated stats and a static GitHub page, which is what the homepage at tintinweb.github.io/smart-contract-sanctuary is. The chain-specific contracts directory holds the sources. And the chain-specific utils directory holds support scripts.

The top level of the repository is correspondingly thin, with the .gitmodules file, the .github directory, the _docs and _utils directories and the nine chain directories that are submodule mount points.

The design is a sensible one for a dataset of this size, since a single repository with tens of thousands of Solidity files would be painful to clone and impossible to update per chain, and the submodule split lets each chain be fetched with a shallow checkout.

Cloning it is a 2 GB operation, and the README says so

The usage section is short, and it comes with a warning: expect a full, recursive check-out to take 2GB+ disk space. The repository is configured for use with git+ssh, which the README describes as much more stable and faster.

The first time, you clone the index and check out all the chain-specific sub repositories from scratch:

git clone --recursive --depth=1 [email protected]:tintinweb/smart-contract-sanctuary.git

If the repository exists but the submodules were never initialised, the README gives a separate command, with the --init and --remote flags and a shallow depth, which pulls the submodules rather than just the index:

git submodule update --init --remote --depth=1 --progress

And if the submodules are already there, the update is the same command without the init, which is what you run to bring the contract data forward:

git submodule update --remote --progress

The depth of 1 in all three is what keeps the operation from being much worse than 2 GB, since a full history of every chain's contract dumps would be far larger. The cost of that choice is that the history of the data is not available, so a question about when a given contract was added to the corpus cannot be answered from the git log.

The README also points at the git submodules documentation for more options, which is the right place to look if you need to fetch only one chain, because fetching all nine is the default and is the expensive part.

The file layout is the interface

There is no database and no manifest you can rely on for a complete list. The README is explicit about that: some contracts are listed in contracts.json, but the file system may contain more files than what is listed in this summary, and you should rely on the folder and file structure for a full list.

The structure itself is simple and deliberate. Contract sources for the various networks are grouped by the first two chars of the contract address, so the directory tree is an index by address prefix, and files are named in the format address underscore source unit name, with the example given being 0f0c3fedb6226cd5a18826ce23bec92d18336a98_URToken.sol.

That naming is what makes the corpus usable. A file whose name starts with an address can be found without an index, the prefix grouping puts related contracts near each other, and the unit name suffix tells you what the source unit declares itself to be, which is the piece of information a security researcher usually wants first.

The one historical note here is that the repo used to auto submit contracts to 4byte.directory, which is a hint about how the data was sourced and enriched rather than simply copied.

The utils directories, by contrast, are where the harvesting logic lives, with support scripts for activities like dumping smart contracts from public sources, and they are documented with a requirement of pip install -r requirements.txt. The development notes explain the update workflow: to use the list of verified contract addresses with an open source license from Etherscan, you download the CSV, add it to the util folder, and run parse_download_contracts_etherscan_io.py with your Etherscan API, which adds the new contracts to the appropriate folder.

So the corpus is reproducible by hand, with an API key and a script, and the reason it stopped is that somebody had to run that loop for nine chains and stop doing it.

You can search it without cloning it

The data science tools section points at three ways of working with the corpus that do not involve a 2 GB checkout, and the difference in approach matters for anybody planning to use the data.

Sourcegraph is described as a powerful online code search service that can be used to search the sanctuary without cloning, and the link in the README is a scoped query against the repository with a path filter for the mainnet directory and a literal pattern. That is the path for a quick answer to a question about one contract.

SolGrep is a scriptable semantic grep utility for Solidity, described as crunching numbers, finding specific contracts and extracting data. The word scriptable is the important part. A grep finds text, and a semantic grep understands what the code means, which is the difference between finding a file that mentions an address and finding the contract that implements a particular pattern.

Semgrep is the third, and the README notes that it now supports Solidity, with thanks given to the person who contributed the Solidity parser. Semgrep is a static analysis tool for finding bugs and enforcing code standards at editor, commit and CI time, and applying it to this corpus turns a data set into a research instrument, because you can run a rule across every verified contract ever published on a chain.

The three tools cover the three levels of work. Search for something you know is there, extract something you need at scale, and analyse the corpus for patterns. None of them needs the clone, and all of them work on the frozen window of 2023 to mid-2024, which is the real constraint on any of them.

No licence, which is the awkward part of a dataset

The repository metadata does not classify a licence, and the README does not state one. For a codebase that is a minor detail. For a dataset that other people are invited to cite in a paper, it is a genuine problem, because the citation section tells you how to attribute the work but not what you are permitted to do with it.

The comparison with the recommended alternative is instructive on this point as well as on freshness. The README points at Sourcify and its open, freely available repository for a larger and continuously updated dataset, and the framing there is that the data is open and freely available. The sanctuary, by contrast, is a set of third-party contract sources, verified on a public explorer, that the project gathered and organised, and the terms under which that corpus may be redistributed are not stated anywhere in the repository.

There is also a question about the sources themselves that the repository does not engage with. These are contracts whose source code was published, usually because a project wanted it verified on a block explorer, and publishing source for verification is not the same as publishing source for redistribution. The project is careful in other respects, noting that the folder and file structure is the authoritative list rather than the summary file, and that the data science tools are for searching and analysing the corpus. But it is a corpus of other people's code, and a team planning to redistribute it, or to feed it into a commercial tool, has a question to ask the author that this README does not answer.

For most research uses, reading and analysing the corpus locally is the low-risk path, and the citation entries are there to make that possible.

Editorial conclusion

The smart-contract-sanctuary still works as a static corpus of verified contract source code, and it is a reasonable thing to diff, grep and study for a specific chain and a specific period. It is not a data source for current deployments, because anything verified after mid-2024 is simply absent, and the README says so plainly. Before you build a tool on top of it, check the licence situation, which is not stated, and check whether the period it covers matches the contracts your research actually needs, rather than assuming the top of the list is the current state of the ecosystem.

Frequently asked questions

Official sources

1. Issues
2. Project website
3. README
4. Releases
5. tintinweb/smart-contract-sanctuary on GitHub