Shoryuken: an Amazon SQS thread based worker whose options live in the wiki

A thread based processor built on the AWS SDK for Ruby

Shoryuken describes itself as 'a super efficient Amazon SQS thread based message processor for Ruby', and it targets one situation in particular: an application that already keeps its queue in Amazon SQS and needs a separate Ruby process to drain it. Topic tags name the moving parts without explaining any of them, with `aws-sdk-ruby` and `aws-sqs` for the transport, `concurrent-ruby` for the threading, `activejob` for the Rails bridge and `sqs-commands` for the command line side.

Nine bullets sit under the heading for key features. Queue Load balancing, Concurrency per queue, Long Polling, Batch processing, Auto extend visibility timeout, Exponential backoff, Middleware support and Rails Active Job are all there, and every one is a hyperlink into the wiki rather than a description of behaviour. Ruby 3.0 or greater is the only requirement stated anywhere in the project.

Nine feature links and no configuration file in the checkout

Skim the repository tree and the shape of the project is clear from the top-level entries alone: `.github/`, `lib/`, `spec/`, `bin/`, `examples/`, `test_workers/`, `Rakefile`, `shoryuken.gemspec`, `Gemfile`, `Gemfile.lint`, `Gemfile.lint.lock`, `.rubocop.yml`, `.yard-lint.yml`, `.rspec`, `renovate.json` and `.ruby-version`. No docs directory and no wiki checkout sit among them, so none of the prose that defines the options ships with the gem.

Option names survive only inside link anchors. Batch processing points at `Worker-options#batch`, extending the visibility timeout points at `Worker-options#auto_visibility_timeout`, and exponential backoff points at `Worker-options#retry_intervals`. Those three anchors give a worker author the configuration keys they may set, and not one default value, batch size, timeout length or retry interval anywhere in the checkout. Concurrency per queue links to a page named Processing Groups, load balancing links to an options page, and between the two links a reader learns that both features exist and nothing about how either is spelled.

Two example files sit in `examples/`, `examples/bootstrap_queues.rb` and `examples/default_worker.rb`. Their names cover the two halves of a setup, creating queues and running a worker, and they are the closest thing in the checkout to a worked configuration, yet no link to either appears in the README.

The single feature with no wiki link is the Amazon SQS CLI. One line points at `shoryuken help sqs` and stops there, so no flag, no output format and no example of starting a worker appear in the repository.

Adding the gem, then running the specs on Ruby 3.0

Installation takes two steps. Add the gem to the application's Gemfile:

gem 'shoryuken'

Then resolve the bundle:

$ bundle

Nothing else follows. Ruby 3.0 or greater is the stated floor, the repository keeps a `.ruby-version` file, and the README never says which Ruby that file pins.

For a first real run, the committed test suite is the only complete workflow written down. Unit specs run against the latest dependency versions:

bundle exec rake spec

Integration specs, which include the Rails tests, need a queue server started first, and the README gives both commands:

docker compose up -d

bundle exec rake spec:integration

Usage then stops. No command for launching a worker against a live queue appears anywhere, and the section called Usage is one link to the Getting Started page in the wiki.

ElasticMQ 1.7.1 on port 9324 replaces Amazon SQS in tests

Talking to the real service is the one thing the repository deliberately swaps out. `docker-compose.yml` defines exactly one service:

services:
  elasticmq:
    image: softwaremill/elasticmq-native:1.7.1
    container_name: elasticmq
    ports:
      - "9324:9324"

Container name, pinned image tag 1.7.1 and the port mapping 9324:9324 are committed values, so the integration suite talks to an SQS compatible server a developer can start and stop locally instead of a queue inside a real AWS account. There is no volume, no environment block and nothing else in that file, which makes it a fixture rather than a development stack. The difference matters precisely when a test fails, because behaviour belonging to the managed service never enters the suite, and no second compose file or endpoint switch exists for sending the same specs to production SQS. The image tag is written into the file by hand, and no instruction for updating it appears in the checkout.

Two Gemfiles, a lint lock file and a pinned Ruby version

Maintenance tooling is the loudest part of the tree. A `.github/` directory, a `Rakefile`, `.rspec` for RSpec, `.rubocop.yml` for style, `.yard-lint.yml` for documentation linting and `renovate.json` for dependency updates all sit at the top level, and the README describes none of them, so each file has to be opened to learn what it governs.

Two bundler setups live side by side. `Gemfile` covers the gem itself, while `Gemfile.lint` is a separate bundle with its own committed lock file in `Gemfile.lint.lock`. Installation instructions point only at the application's own Gemfile and never mention the lint bundle, which is worth knowing before either lock file is edited.

The last push to the default branch main landed on 2026-09-28, and the repository is not archived, so this layout is a working one rather than a frozen one.

A version bump is two hand edits and one rake task

The tagging procedure is spelled out step by step, which few README files bother with. Comparing the newest tag against HEAD comes first:

git log $(git describe --tags --abbrev=0)..HEAD --oneline

Releases are cut by hand from there: someone updates `CHANGELOG.md`, edits `lib/shoryuken/version.rb` with a SEMVER version number, and runs a rake task.

bundle exec rake release

Three separate steps sit between that comparison and the task, and the one file the README names as holding the number is `lib/shoryuken/version.rb`. The contributing section has the same shape: five numbered steps that fork the project, create a feature branch, commit, push and open a pull request, with no review or test requirement stated.

Where a thread pool is the wrong shape for the work

Several constraints make this the wrong tool in specific cases. An application that cannot reach Ruby 3.0 or greater is ruled out by the stated floor, and a queue that is not Amazon SQS is ruled out by construction: `aws-sdk-ruby`, `aws-sqs` and `sqs-commands` are the transport topics, and no other backend appears in the tree.

Threading brings limits the README does not quantify. Concurrency per queue and load balancing both point at separate wiki pages, so process and thread counts are configuration living outside the checkout, and no default value is stated for either. Batch processing under `batch` and visibility timeout handling under `auto_visibility_timeout` are the two options most exposed to load, and both are named without a number.

Deployment help is missing as well. There is no Dockerfile for the worker, no process manager definition and no scheduler example among the top level entries; the only compose file exists to start the test queue server on port 9324. Long polling, middleware support and exponential backoff get one line each, so a team that needs to know how retries interact with middleware ends up reading the wiki. `shoryuken help sqs` is the only command interface named at all, with no output, exit code or permission scope documented for it.

Editorial conclusion

Shoryuken fits teams already storing their queue in Amazon SQS and already on Ruby 3.0 or greater, and it fits them badly if they need a different backend, documented option defaults or a deployable worker image. Before adopting, read the pages reached from `Worker-options#batch` and `Worker-options#auto_visibility_timeout`, since the checkout names those settings and defines none of them, then confirm credentials, region and access keys against the real service rather than against ElasticMQ 1.7.1 on port 9324.

Frequently asked questions

Official sources

1. Issues
2. README
3. Releases
4. ruby-shoryuken/shoryuken on GitHub