PgParty gives ActiveRecord a first class idea of a PostgreSQL partition

What the gem adds to a migration

PgParty provides ActiveRecord migrations and model helpers for creating and managing PostgreSQL 10+ partitions, and the feature list is short and precise. There are migration methods for partition specific database operations, and model methods for querying partitioned data, creating adhoc partitions, and retrieving partition metadata.

The migration methods are the larger half, and they come in families. There is a create method for each partition strategy: create_range_partition, create_list_partition, and create_hash_partition for version 11 and later. Each takes a table name and a partition key, and each can have a partition of it created underneath with the corresponding create_..._partition_of method, which for range takes a start and end range, for list a set of values, and for hash a modulus and a remainder.

Then there is the default partition method, create_default_partition_of, for values that fall outside the range or list constraints of every other partition, and the attach methods, which attach an existing table to a partitioned table rather than creating a new one. The attach methods are the migration path for the common case of a table that already holds data.

So a partition lifecycle in this gem reads as: create a partitioned parent, create or attach the children, and optionally create a default partition to catch anything the others miss. The methods are available both in migrations and on ActiveRecord::Base#connection objects, which means the same call works in a migration and in a console or a task.

The option set, and why primary keys are off by default

The configuration is reached through PgParty.config and PgParty.configure, and six settings are documented.

Caching is on by default, and it covers currently attached partitions and anonymous model classes, with a caching_ttl that defaults to -1, meaning cache entries never expire. schema_exclude_partitions defaults to true and controls whether child partitions are excluded in rake db:structure:dump. create_template_tables defaults to true, and the template: option when creating a partitioned table overrides it. include_subpartitions_in_partition_list defaults to false for backward compatibility, and can always be overridden with the include_subpartitions: option.

The one to read carefully is create_with_primary_key, which defaults to false, and the reasons are stacked. The behaviour is disabled by default because the configuration usually requires composite primary keys to be specified, and ActiveRecord does not natively support composite primary keys. There are workarounds such as the composite_primary_keys gem.

Three more constraints follow. It is not supported for Postgres 10, because it requires Postgres 11 or later. Primary key constraints must include all partition keys, with the example given being a primary key of an id and a created_at with a created_at partition key. And partition keys cannot use expressions.

So the default is off because the correct configuration is often impossible inside ActiveRecord without a workaround, and a partition key that cannot be an expression is a genuine limitation of the approach rather than of the gem.

Caching is per process, and that changes the deployment advice

The caching note is the one that has operational consequences, and it is easy to read past. Caching is done in-memory for each process of an application. Attaching or detaching partitions will clear the cache, but only for the process that initiated the request. For multi-process web servers, it is recommended to use a TTL or disable caching entirely.

That recommendation is the important part. A Puma or Passenger deployment runs several processes, each with its own copy of the in-memory cache, so a partition attached by one worker is not visible to the others until their caches expire. With the default TTL of -1, meaning never expire, a stale cache in a worker that did not handle the attach persists until something else clears it.

The example configuration shows the shape of a sensible setup, in a Rails initializer:

# in a Rails initializer
PgParty.configure do |c|
  c.caching_ttl = 60
  c.schema_exclude_partitions = false
  c.include_subpartitions_in_partition_list = true
  # Postgres 11+ users starting fresh may consider the below options to rely on Postgres' native features instead of
  # this gem's template tables feature.
  c.create_template_tables = false
  c.create_with_primary_key = true
end

A 60 second TTL is a reasonable compromise between the cost of re-reading the partition list and the window in which a newly attached partition is invisible to other workers. Disabling caching entirely is the safe option when partition changes are frequent, since the cost is a query per operation rather than per request.

The one limitation that changes your schema format

There is a single limitation, and it is stated at the top of the readme rather than buried.

Partition tables are not represented correctly in db/schema.rb, so you should use the SQL schema format.

That is a bigger change than it looks. The Ruby schema format is the default in a Rails application, and it captures the schema as Ruby code rather than as SQL, which lets you diff and load it without a database connection. The SQL format dumps the schema with the structure dump command, which is what the gem's schema_exclude_partitions option interacts with, since that option controls whether child partitions are excluded in rake db:structure:dump.

So the choice of schema format is now a requirement rather than a preference for any application that wants to use this gem. The upside is that the SQL format is closer to what the database actually holds, which is arguably the right thing for a partitioned schema where the Ruby format has no concept of a partition. The cost is that the schema is now a database artefact rather than a Ruby file, and the two formats are not interchangeable in a project of any size.

The gem also does not require the pg gem, since some model methods may work for other databases, and the documentation notes that migration methods will be unavailable unless pg is installed. So a project could use the model helpers on a different database and drop the migrations entirely.

Compatibility, and a plan to move the code elsewhere

The compatibility matrix is specific and current. The gem is tested against Rails 7.2, 8.0 and 8.1, Ruby 3.2 and the latest, which is currently 3.4 at the time of the commit, and PostgreSQL 14, 15, 16, 17 and 18.

Note what is not in that list: PostgreSQL 10 and 11 are not tested, even though the gem supports them, with the hash partitioning methods and the default partition explicitly marked as version 11 and later. And PostgreSQL 13 is skipped between 12 and 14 in the tested range, which is consistent with a project that tests the versions still receiving upstream support.

The future work section is honest about where this is going. The plan is to separate out the model functionality into a new gem and port the migration functionality into pg_ha_migrations, with some of that already done, and the author states that the current gem will continue to be maintained for bugfixes and support for new Rails versions until that work is complete.

The author also explains a decision not to build something. Automatic partition creation was originally planned, but the judgement is that the functionality would be better served by pg_partman, the established PostgreSQL extension for the job.

The repository shows a conventional Ruby gem with the tooling to match: an Appraisals file for testing against multiple gem versions, a gemfiles directory, a .rspec file, a .pryrc, a Rakefile, a bin directory, a Dockerfile, a docker-compose.yml and a spec directory. The Docker setup runs against a PostgreSQL image with a configurable version, defaulting to 18, and installs the matching postgresql client.

Editorial conclusion

Adopt PgParty if you run ActiveRecord on PostgreSQL and want partitioning expressed as migration methods and model calls rather than raw SQL, and you are already using the SQL schema format. Do not adopt it if your application is still on the Ruby schema format, because the gem states plainly that partitioned tables are not represented correctly in db/schema.rb. Before you start, check three things: that you have switched to the SQL schema format, whether your PostgreSQL version supports the partition method you need, since hash partitioning and the default partition need version 11 or newer, and how the primary key on a partitioned parent table will work, since ActiveRecord does not support composite primary keys natively.

Frequently asked questions

Official sources

1. Issues
2. License: MIT
3. README
4. rkrage/pg_party on GitHub