sift: configurable grep alternative for code and log analysis

Why sift treats search configuration as a repeatable policy

sift addresses recurring problems in code search, log investigation, digital forensics, and data processing. Its primary design choice is persistent configuration. Instead of requiring a long command every day, a user can record choices such as case-insensitive matching, line numbers, binary-file skipping, and Git ignore handling in a configuration file. That setup is relevant to source-code work, where the same audit may be repeated across repositories.

The project combines file discovery with matching. According to the README, sift searches everywhere when no filtering has been configured, but users can add custom file types or write a config that restricts a run. This makes it useful when the file extension is part of the question rather than merely a filter around a text match. The documented audit that limits a search to PHP files demonstrates this approach.

This is not presented as a drop-in replacement for every part of GNU grep. The distinguishing feature is a reusable rule set around a search. Users who only need a single literal lookup may find ordinary grep simpler. sift makes more sense when repeated searches need consistent treatment of ignored files, binary data, file types, line presentation, and nearby context.

sift separates options, matching, and result presentation in Go

Based on the repository layout, sift separates its command implementation into recognizable Go components. `sift.go` contains the application entry point, `options.go` handles option definitions, `matching.go` and the architecture-specific matching files contain search behavior, and `output.go` handles result presentation. File-type and ignore-rule support is also visible through `filetypes.go` and the `gitignore/` directory. This points to an application in which configured options influence matching and output while file-type and ignore rules constrain which paths are considered.

The dependencies in `go.mod` add limited evidence about that design. It declares `github.com/svent/go-flags`, `github.com/svent/go-nbreader`, `golang.org/x/crypto`, and `golang.org/x/term`, but the source does not establish the exact buffering sequence or identify the sift features that use each dependency.

Sift includes separate files for `matching_amd64.go`, `matching_amd64.s`, and `matching_other.go`. This indicates a distinct AMD64 implementation with a fallback for other systems. Engineers reviewing the source can start with option parsing, then follow matching and output. The README does not document internal APIs, extension points, or a plugin mechanism, so the public command and configuration surface remain the confirmed areas for evaluation.

Saved settings and contextual conditions shape sift's searches

The documented configuration command uses `--write-config` to turn command-line choices into a saved setup. The command enables case-insensitive matching, line numbers, binary-file skipping, and `.gitignore` processing. Its main implication is that later searches can inherit these decisions rather than require all four flags every time. The README also states that configuration can be overridden for specific directories, which provides a default rule set with narrower local exceptions. It does not show the override syntax, so engineers must consult the linked documentation before scripting directory-specific behavior.

Sift's conditions extend matching beyond a query and its immediate line. The documented PHP audit combines several mechanisms: `-x php`, the `mysql_query` search term, `--preceded-within "5:_(GET|POST)"`, and `--not-preceded-within "5:escape"`. As a policy check, the query seeks PHP files where the call is close to external-input notation but is not close to a call to an escaping function. The five-line windows allow an audit-shaped question without an external script to reconstruct context.

This condition model is the strongest project-specific reason to consider sift, but it also creates a configuration burden. A saved config can enforce consistency across a team, while a complicated condition can be hard to review if its rules are not centralized. The README does not provide a formal condition grammar or explain conflicts among overlapping conditions. Teams should inspect actual config files before standardizing them.

Installing sift and running its first documented configuration

The README lists binaries for Linux, Windows, OS X, and BSD on the project download page. It also documents Homebrew installation on OS X. This command should install the `sift` executable from the documented Homebrew package:

brew install sift

On Arch Linux, the README presents package repository options through `yaourt`. The `sift-bin` package downloads and installs a binary:

yaourt -S sift-bin

The README also gives a source package option:

yaourt -S sift

For users with a working Go environment, the documented installation command is:

go get github.com/svent/sift

The module identifies itself as `github.com/svent/sift` and declares Go 1.23.4 in `go.mod`.

A first useful setup is the documented persistent configuration command:

sift -i -n --binary-skip --git --write-config

This tells sift to ignore case, show line numbers, skip binary files, respect `.gitignore`, and write the configuration. The README does not state the generated filename or directory, so users should verify that behavior against the linked documentation before automating first-run setup. A concrete audit can then follow the documented condition example:

sift -x php mysql_query --preceded-within "5:_(GET|POST)" --not-preceded-within "5:escape"

This query should limit matching to PHP files and apply the two five-line context conditions. The README does not include sample output, so it does not support claims about the exact result layout.

Large-file claims leave performance and scope questions open

The README states that sift has been tested successfully with files larger than 50 GB, more than 5,000,000,000 lines, and more than 5,000,000,000 matches. Those are project-reported cases, not a promise about performance on every filesystem, query, or machine. The documentation does not publish hardware details, elapsed times, memory use, storage type, or test procedures for those cases. The numbers establish a large-file use case but cannot support a general throughput guarantee.

The default search scope has a direct operational cost. The README says sift searches everywhere unless configured otherwise. For an analyst examining logs or external data in a dedicated directory, that broad default may be useful. In a large home directory, mounted volume, or mixed content tree, it can increase the number of paths considered. Adding file types and ignore rules reduces that scope, but the operator has to decide which sources are relevant.

Binary skipping is available, yet it is not enabled until the user adds `--binary-skip`. Code search can also create false confidence if a pattern finds tokens without understanding application behavior. The PHP condition example is more discriminating because it checks nearby context, but it remains a textual audit and does not establish exploitability. Sift reports textual relationships, not semantic reachability or vulnerability confirmation.

GNU grep is simpler when sift's saved policy is unnecessary

The clearest alternative is GNU grep. A user who needs a literal string, regular expression, line numbers, or a basic recursive search can use a familiar tool without first writing a sift configuration. Sift differentiates itself through safe defaults, persistent settings, `.gitignore` handling, custom file types, multiline support, and contextual conditions. The README does not claim complete GNU flag compatibility.

That difference matters for scripting. Moving an existing grep command to sift may require reviewing supported flags and output behavior rather than substituting the executable name. The README provides no compatibility table, migration guide, or inventory of GNU-specific options. Engineers should classify commands into two groups: simple one-off searches that can remain in grep, and repeatable source or log audits that benefit from sift's saved filtering and context rules.

Maintenance status is recent according to the repository record. The last push was on 2026-06-30, and version v0.9.2 was released on 2026-06-30. Earlier versions were v0.9.1 on 2025-09-30 and v0.9.0 on 2016-10-23. That history shows a long gap followed by two recent releases. The repository is not archived, but the README does not describe a support policy for older Go versions, operating systems, or release transitions.

Sift is distributed under GPL-3.0. The README states that redistribution and modification are allowed under that license's terms and directs users to the GNU GPL text. This review does not interpret those obligations. Organizations should check GPL-3.0 compatibility for their distribution model before incorporating or repackaging the software.

Editorial conclusion

sift fits developers and data analysts who want a configurable search tool for source trees, logs, or large files. It is less suitable for users who need documented GNU grep compatibility or prefer a simple command with no saved configuration. Before adoption, verify whether its persistent .gitignore behavior, file-type selection, and contextual conditions match the intended workflow, and confirm how its GPL-3.0 licensing affects your distribution model.

Frequently asked questions

Official sources

1. License: GPL-3.0
2. Project website
3. README
4. Releases
5. svent/sift on GitHub