Open-source project
phiresky/sql.js-httpvfs avatar
phiresky/sql.js-httpvfs

sql.js-httpvfs: a 1 KiB page size, a cacheBust token, and a virtual file system with no tests

GitHub describes it as Hosting read-only SQLite databases on static file hosters like Github Pages. The repository metadata lists TypeScript as its primary language. The metadata lists the Apache-2.0 license. This article stays within the project description and details documented in the GitHub repository README.

3,711 stars108 forksTypeScriptApache-2.0

At a glance

What is it?
sql.js-httpvfs lets a browser query an SQLite database hosted on a static file hoster, fetching only the HTTP ranges it needs. The design pushes the hard work onto your schema, prescribing a 1 KiB page size and careful index order, and it ships a cacheBust token because a stale CDN chunk can corrupt the database rather than merely lag it.
Who is it for?
sql.js-httpvfs fits a read-only dataset you want to ship as static files and query in the browser, small enough that the request pattern does not matter much. Four things to weigh before building on it.
Can I use it commercially?
Yes. Apache-2.0 is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
Is it still maintained?
Probably not. The repository last received commits 27 months ago, on August 6, 2024.
What is it written in?
Mainly TypeScript, according to GitHub's language statistics.

Answers come from the project's GitHub data, last synced on November 15, 2026, and from our analysis. They are not legal advice.

From daily snapshots

Star history

  • phiresky/sql.js-httpvfs3,711 ★ · +15
Star history
Repository2026-08-132026-11-15
phiresky/sql.js-httpvfs3.7k3.7k

15 Stars in the last 94 days, measured from the daily GitHub snapshot. Compare with other projects

Editorial analysis

It works only if the database is prepared for it

The performance claim carries a condition, stated in one line: the whole thing only works well if the database and indexes are structured well, and having many correct indices matters even more here than in a normal database. The preparation recipe is four statements. Journal mode is set to delete, with the comment that this is what makes it possible to set the page size at all, which means a database hosted this way cannot be in WAL mode. The page size is then set to 1024 bytes, described as a trade-off between the number of requests that have to be made and the per-request overhead, and it is a far smaller page than a database would normally use. Each full text search table is optimized, and a vacuum reorganizes the file and applies the new page size.

sql
-- first, add whatever indices you need. Note that here having many and correct indices is even more important than for a normal database.
pragma journal_mode = delete; -- to be able to actually set page size
pragma page_size = 1024; -- trade off of number of requests that need to be made vs overhead.

insert into ftstable(ftstable) values ('optimize'); -- for every FTS table you have (if you have any)

vacuum; -- reorganize database and apply changed page size

The order matters. Journal mode has to be set before the page size can be, and the vacuum is what actually rewrites the file at the new size, so a database that was created at the default page size cannot be fixed by editing the pragma later.

The published package is the build output and nothing else

The manifest declares version 0.8.12, an Apache-2.0 licence, an entry point at `dist/index.js` and a published file list of `dist/*`. So what npm delivers is the bundled output, not the sources. That matters because the README tells readers the virtual file system logic could be useful to other applications and names the file to read, `src/lazyFile.ts`, which is not in the package. Anyone planning to reuse that code has to clone the repository rather than add a dependency. The dependency list is otherwise unusually small: exactly one runtime dependency, comlink, which is the worker communication layer. Everything else sits in devDependencies, including the webpack toolchain, TypeScript and the type packages. There are no GitHub releases, so the manifest version is the only version reference a consumer has.

A stale cache chunk corrupts the database, not just the page

One configuration property deserves its own attention because of how it fails. Alongside the URL or URL prefix, config accepts an optional `cacheBust` value that is appended as a query parameter to the URLs the library requests. The stated purpose is cache eviction avoidance, and the instruction is to set it to a random value whenever the database is updated, so that you avoid caching-related database corruption. Corruption is the right word for the failure mode: if a CDN still serves an old chunk while the manifest and other chunks come from a new build, the page data no longer agrees with itself. The same advice applies to a remote JSON config, identified by the `from: 'jsonconfig'` value, which needs its own busting value or the config will outlive the database it describes.

The debugging advice tells you to leave the browser

The section on debugging data fetching is the most practically useful part of the file, and its central instruction is counter-intuitive. Start with the query plan: a plain table scan means the table is downloaded pretty much in full, a scan using an index means index lookups followed by table lookups by rowid, and a scan using a covering index means the index alone answers the query with no table lookup at all, which is the fastest case. Index column order matters, with the columns that reduce the result count most listed first, and the file gives a recipe for an index containing exactly the filtered and selected columns so SQLite can read it sequentially. Then it says to look at the `dbstat` virtual table to see which pages were read, using the log line format and `worker.getResetAccessedPages()`. And to do that in an SQLite3 shell rather than the browser, because `dbstat` reads the whole database, which is the one query that defeats the library.

A read-only library that also ships a writable DOM table

The project describes itself as a read-only HTTP range based virtual file system, and the summary line says read-only databases. Two other capabilities sit alongside that framing. The same file notes that the lazy fetching might be worth implementing as a SQLite virtual file system directly, so that SQLite could be compiled with something like the WASI SDK instead of relying on the whole Emscripten operating system emulation, which is a roadmap item rather than a feature. More concretely, the repository also provides what it calls a proof-of-concept level implementation of a DOM virtual table, which allows interacting with the browser DOM directly from inside SQLite queries, and that interaction is described as read and write. So the read-only guarantee belongs to the database file, not to everything the library can do.

The author says the virtual file system part has no tests

There is a section asking whether the library is production ready, and the answer is unusually direct. It was mainly written for small personal projects and as a demonstration, and the author notes having received requests for applications outside its scope. In general it works, but there is no effort to support older or unusual browsers, and if the browser lacks WebAssembly and Web Workers it will not run at all. There is no cache eviction, so the more data a session fetches the more memory it uses, which is a property worth understanding before pointing it at a large database. The most important sentence is about testing: most of the complicated work is done by SQLite, which is well tested, but the virtual file system part does not have any tests. The last commit to the default branch is dated 6 August 2024.

The alternatives are named by the author, and one is a rewrite

For anything larger, the file recommends looking elsewhere and names three specific places. The first is a discussion of a general virtual file system in the upstream sql.js tracker. The second is wa-sqlite, described as a much simpler WebAssembly wrapper with different virtual file systems that do not require an Emscripten dependency, and the author adds that this library could easily be reimplemented on top of it. The third is absurd-sql, an implementation of a virtual file system that allows persistence and read and write queries by storing the database in IndexedDB, which is the option when the data has to change. The inspiration section points the same way: a torrent-based virtual file system for SQLite is described as theoretically even better than an HTTP one, but limited to native SQLite, with a request in the open for someone to port it and pair it with a WebTorrent transport.

Webpack 5 with a webpack 4 dev server and a nightly compiler

The dependency versions tell you when the project was last assembled. The bundle is built by webpack 5, but the development server is pinned to the 3.x line, which belongs to webpack 4. The TypeScript dependency is a caret range whose floor is a dated development build rather than a release, so the lower bound of the allowed range is a prerelease. The type packages include typings for the upstream sql.js package even though this repository is a fork of it, and typings for React, React DOM and a plotting library that the library itself does not use. The tree shows the shape of the experiment: the upstream sql.js sources vendored in a directory of their own, a `create_db.sh` script at the root for splitting a database into chunks, a data file for benchmarking, a root `index.html`, and an example directory that ships its own database file, package manifest, tsconfig and webpack config, so the example is a self-contained application rather than a snippet.

Editorial conclusion

sql.js-httpvfs fits a read-only dataset you want to ship as static files and query in the browser, small enough that the request pattern does not matter much. Four things to weigh before building on it. The library works only when the database is prepared for it, with a 1 KiB page size, journalling switched off WAL-style, and indexes ordered so the filtering columns come first. The published package contains the built bundle only, so the file the project points you at for reuse is not in the package you install. Updating a database requires changing the cacheBust token or you risk mixing cached and fresh chunks, which the author describes as caching-related corruption. And the author states plainly that the library was written for small personal projects, that the virtual file system part has no tests, and that there is no cache eviction.

Frequently asked questions

Can I use SQL with JavaScript?

This repository is one answer to that. It is a fork and wrapper around sql.js, which is SQLite compiled to WebAssembly, and it adds an HTTP range based virtual file system so a browser can query a database hosted on a static file hoster without downloading the whole file. It has one runtime dependency.

What are the downsides of this approach to SQLite in the browser?

The file names its own. It only works well if the database and indexes are structured well, so the page size is set to 1024 bytes and journal mode is set to delete. It needs WebAssembly and Web Workers, it supports no older browsers, there is no cache eviction so memory grows with the data fetched, and the virtual file system part has no tests.

Is a SQLite database served to the browser read-only here?

The database file is treated as read-only, since fetching is by HTTP range. The library also ships what it calls a proof-of-concept DOM virtual table that allows reading and writing the browser DOM from inside SQLite queries, which is a separate capability from the hosted file.

Official sources

  1. Official README
  2. Project repository
  3. README
  4. Releases
Add this badge to your README

If you maintain this project, the badge below links readers to this analysis and shows its maintenance status from the daily GitHub snapshot. Paste the markdown into your README; add ?metric=license or ?metric=stars to the image URL for a different field.

Add this badge to your README

markdown
[![Hysen Labs](https://hysenlabs.com/badge/phiresky-sql-js-httpvfs.svg)](https://hysenlabs.com/projects/phiresky-sql-js-httpvfs)

Star history chart for your README

markdown
[![Star history](https://hysenlabs.com/chart/phiresky-sql-js-httpvfs.svg)](https://hysenlabs.com/projects/phiresky-sql-js-httpvfs)

Was this useful?