gammazero/workerpool: Non-Blocking Go Concurrency Limiting

Why workerpool separates execution limits from queue limits

gammazero/workerpool solves a narrower problem than a conventional worker pool. Its documented purpose is to limit the concurrency of task execution, while the number of tasks waiting to run remains unbounded by the library. Submitting a task therefore does not block, even when the configured execution limit has been reached. This makes the package relevant to Go programs that need to preserve the responsiveness of producers while controlling how many submitted functions run at once.

The distinction has direct design consequences. A pool with two workers can receive many more than two tasks. Two may execute concurrently, while additional tasks wait for execution capacity. The README does not promise a maximum queue length, flow control, admission control, persistence, or automatic distribution across processes. Those concerns remain with the application or its surrounding infrastructure.

This behavior is useful when a producer receives inbound work faster than it can be processed but cannot tolerate blocking at the submission point. It can also help prevent an unbounded number of application goroutines from performing the same class of work simultaneously. However, non-blocking submission moves pressure into memory rather than applying backpressure. The package should therefore be selected for a specific concurrency boundary, not as a general-purpose queue with predictable resource usage.

How workerpool.New, Submit, and StopWait organize work

The public example uses workerpool.New(2), which creates a pool configured for two concurrent workers, then calls wp.Submit with a function for each request. After all submissions, it calls wp.StopWait(). The API presents a small surface: create the pool, submit tasks, and wait for shutdown when the caller needs outstanding work to finish.

The repository layout includes workerpool.go and options.go, plus a separate pacer directory. That layout indicates that the core pool and configurable behavior are separated in the source tree. The README does not describe the internal queue type, goroutine lifecycle, worker startup sequence, or exact policy used when workers become available. Those implementation details should be checked in the source before relying on an assumption such as FIFO execution or a particular scheduling guarantee.

The README explicitly contrasts this package with gammazero/workers, which it describes as a pool that blocks when all workers are busy. That difference is the central architectural choice. workerpool accepts work without waiting for a free worker, while the alternative applies backpressure at submission time. Neither approach is universally preferable. The first fits a non-blocking producer contract, and the second fits a system that must keep queued work bounded.

Installing the package and submitting the first batch

The README gives one installation command for use in an existing Go workspace. It does not provide a separate initialization command or application scaffold, because workerpool is a library rather than a standalone service. The module path in go.mod is github.com/gammazero/workerpool, and the module currently declares a Go 1.26 requirement.

Run this command in the Go module where the application should depend on workerpool:

$ go get github.com/gammazero/workerpool

The first complete use submits five request values to a pool with an execution limit of two. Each submitted closure captures the loop variable and prints the value it handles.

package main

import (
	"fmt"
	"github.com/gammazero/workerpool"
)

func main() {
	wp := workerpool.New(2)
	requests := []string{"alpha", "beta", "gamma", "delta", "epsilon"}

	for _, r := range requests {
		wp.Submit(func() {
			fmt.Println("Handling request:", r)
		})
	}

	wp.StopWait()
}

The reader should see all five messages eventually printed. The example does not print the internal queue state, so it demonstrates submission and completion rather than proving the exact number of simultaneous executions from output alone. The README also links to an example wrapper function for showing the start and finish times of a submitted function.

The unbounded queue is the primary operational constraint

The README is unusually direct about the package boundary: there is no upper limit on the number of queued tasks other than system resource limits. If inbound work is too numerous even to queue, workerpool does not claim to solve that problem. The documented alternatives are distributing work across multiple systems or storing pending input in intermediate storage such as a file system or a distributed message queue.

That limitation can appear as memory growth, allocator pressure, or an ever-growing backlog. Since submission is non-blocking, a producer may continue receiving requests and adding work even when processing cannot keep pace. The library does not report that condition through the documented example, and the README does not define queue metrics, overflow handling, or a rejection policy. A production application must inspect the package source and tests if it needs those guarantees.

A useful design rule is to treat workerpool as the execution layer after admission decisions have already been made. If a request cannot safely wait in memory, place a bounded queue, durable store, or upstream service between the producer and workerpool. If callers need immediate rejection when capacity is exhausted, use a pool whose contract blocks or fails submission. Choosing workerpool without checking that requirement can turn a concurrency limit into an operational incident.

Comparing workerpool with a blocking workers pool

The README names gammazero/workers as the relevant alternative when all workers being busy should block task submission. The difference is not a claim that one package has more features. It is a different admission policy.

workerpool limits active execution and accepts additional tasks into an unbounded queue. The producer does not wait for a worker, so callers can continue producing work. This favors asynchronous request handling and makes the package suitable when submission latency matters more than immediate queue control.

gammazero/workers is the appropriate direction when the application must stop producing while the pool is saturated. That backpressure can limit work in memory and make overload visible at the point where work enters the system, but it changes the calling contract. Code that assumes Submit will return without delay may need redesign around the blocking behavior.

Neither package description in the README establishes claims about throughput, fairness, cancellation, retry, ordering, or shutdown speed. Those properties must be verified independently before selecting one as a replacement for the other.

Lifecycle and maintenance considerations for a Go dependency

The StopWait call in the README is the documented way to wait after submission when the application needs pending tasks to finish. The repository contains workerpool_test.go, which is a source for checking edge cases, but the README does not document cancellation, task failure handling, or whether a task panic is recovered. Applications should not infer those behaviors merely from the presence of StopWait.

The repository is not archived, and the recorded last push was on 2026-10-12. Recent releases listed in the repository are v1.2.2 on 2026-10-12, v1.2.1 on 2026-02-14, and v1.2.0 on 2026-02-14. The upgrade cost of a small Go library should be evaluated through its module version and API changes rather than through release names alone. The repository declares Go 1.26 and depends on github.com/gammazero/deque v1.2.2, so the consuming toolchain and module graph should be checked before upgrading.

The MIT license permits use under its terms, but that statement is not legal advice. Organizations should review the LICENSE file for obligations that apply to their distribution and third-party context. No separate compatibility policy or upgrade procedure is documented in the README.

Practical boundaries before adopting workerpool

workerpool is a focused answer to one question: how can a Go program submit tasks without blocking while keeping concurrent execution limited? It is not a complete job system. The README does not promise durable processing, retry policy, priority scheduling, cancellation, result collection, rate limiting, or queue-size enforcement. The separate pacer directory may relate to pacing behavior, but the README does not explain its public configuration, so no specific option should be assumed.

For an HTTP handler or message consumer, the important design check is whether the surrounding system can tolerate retained tasks. For a background job that must never be lost, an unbounded in-memory queue is an incomplete solution. For a batch import, the application may already know the complete workload and could prefer a blocking workers pool or its own bounded queue.

The first verification should therefore be API and lifecycle review, followed by inspection of queue growth and shutdown behavior under the intended workload. After that, verify the Go version, dependency versions, and whether the consumer needs backpressure. Once those conditions match, the package is most suitable as a small concurrency boundary around tasks that can safely wait for execution.

Editorial conclusion

Adopt workerpool when submitting goroutines must remain non-blocking and a bounded number may execute concurrently. Do not choose it when the queue itself must be bounded or when backpressure is required, and verify task lifetime, shutdown behavior, and memory cost before integration.

Frequently asked questions

Official sources

1. gammazero/workerpool on GitHub
2. Issues
3. License: MIT
4. README
5. Releases