xitongsys/parquet-go for Go Parquet file reading and writing

Why parquet-go targets Go Parquet file workflows

parquet-go addresses a specific engineering problem: reading and writing Parquet files directly from Go programs. Its README describes the project as a pure Go implementation and lists support for both nested and flat Parquet files. The target audience is therefore Go developers who need file-based columnar serialization without adopting a separate service or non-Go runtime. The library is most relevant when application records already have Go struct definitions and the team wants the field mapping expressed in code rather than duplicated across schema documents.

The README presents struct tags as the main interface. A field name, Parquet type, encoding, repetition, and optional metadata are attached with a `parquet` tag. This is a practical design for batch-oriented jobs, file exporters, and data processing utilities. It is also a limitation for developers who need an unrestricted schema engine or want to avoid coupling their domain structs to Parquet metadata. The repository includes examples for flat and nested local files, CSV conversion, JSON writing, partial reads, type aliases, dates, metadata, and Arrow conversion. That breadth is documented through the example directory, but the README does not provide a single API guide covering every example.

How struct tags become Parquet schemas and data

The documented mapping starts with primitive and logical types. Primitive values include `bool`, `int32`, `int64`, `string`, `float32`, and `float64`, while logical types add `DATE`, timestamps, unsigned integers, intervals, decimals, `LIST`, and `MAP`. A logical type is stored as a primitive type according to the README. Repetition is expressed through the Go field shape: pointers represent `OPTIONAL`, slices can represent repeated values, and the `repetitiontype=REPEATED` tag distinguishes a repeated field from a Parquet `LIST`.

Encodings are selected per field. The README lists `PLAIN` and `PLAIN_DICTIONARY` for all types, delta encodings for selected integer and byte-array types, and notes that not every platform supports every encoding. Dictionary encoding has a direct memory trade-off. The README warns that fields with many distinct values can consume substantial memory because the encoder records values in a map, and that the dictionary index is 32-bit. Large array values can also be duplicated as page minimum and maximum statistics. The documented mitigation is `omitstats=true` in the field tag when those statistics are not useful. This is a library-level memory and file-size decision, not a universal Parquet guarantee.

For `LIST` and `MAP`, the README states that standard formats are converted to Go slices and maps. It also warns that existing Parquet files may use nonstandard layouts. In those cases, parquet-go converts them to corresponding structs instead. That compatibility behavior is useful when ingesting files from varied producers, but it also means an application should inspect the actual schema and field shapes before assuming that every collection will arrive as a native slice or map.

Installing parquet-go and running the flat-file example

The README gives an older GOPATH-oriented installation command, followed by a local example invocation. The repository also contains `go.mod`, with the module path and Go version shown below. A Go module-based project can declare this module path as documented, while the exact module management behavior still depends on the consuming application's toolchain.

The following command installs the library as described by the README:

go get github.com/xitongsys/parquet-go

The repository's `go.mod` identifies the module as follows:

module github.com/xitongsys/parquet-go

go 1.16

The README then directs users to the `example/` directory and names `local_flat.go` as the first local example. From the checkout, the documented invocation is:

cd $GOPATH/src/github.com/xitongsys/parquet-go/example
go run local_flat.go

According to the README, this example creates sample data and writes `example/output/flat.parquet`. The file should be created at that path after the command completes successfully. The example is a better first use than inventing a separate API call because it is the repository's documented path from installation to a real Parquet output file. The README does not document rollback, command-line flags, output inspection commands, or a supported Windows setup for this example.

Supported types, encodings, and compatibility constraints

The type system is broad enough for common analytical schemas, but it exposes the underlying Parquet representation rather than offering an independent Go domain model. For example, `INT96` maps to `string` and is marked deprecated in the README. `DECIMAL` can map to `int32`, `int64`, or string-backed byte representations. Logical timestamps have both converted-type and logical-type forms, with configuration for UTC adjustment and units. Type aliases are supported when the alias base type follows the table.

The encoding list is a compatibility guide as much as a performance choice. `PLAIN` is the safest documented fallback when an encoder is uncertain. `PLAIN_DICTIONARY` is presented as broadly applicable, but the memory warning makes it unsuitable for high-cardinality fields in some workloads. Delta encodings are listed only for particular logical and primitive types. The README also says that some platforms do not support all encoding kinds. It does not publish a compatibility matrix, so adopters must determine whether their producer, consumer, and storage platform accepts the selected encoding.

Compression is represented in the repository structure through the `compress/` directory, and the module includes compression-related dependencies such as `github.com/klauspost/compress` and `github.com/pierrec/lz4/v4`. The README excerpt does not specify which codecs can be selected through public configuration, nor does it document compression trade-offs. That is an important documentation gap for applications whose storage or query engines require a particular codec.

The main trade-off versus Arrow-oriented Go workflows

The repository includes `example/arrow_to_parquet.go` and depends on `github.com/apache/arrow/go/v12`. Arrow and Parquet solve related problems but at different boundaries. parquet-go is centered on reading and writing Parquet files, mapping Go structs through tags, and selecting Parquet encodings. Arrow-oriented Go code is centered on Arrow data structures and their columnar representation, with conversion code for moving between Arrow and Parquet. The presence of the Arrow example confirms that the repository is aware of that interoperability boundary, but it does not document Arrow as a full replacement architecture.

For a service that already uses Arrow tables throughout processing, an Arrow-first approach may reduce conversion work. For a Go utility that only needs to produce or consume a Parquet file, parquet-go offers a more direct path based on structs and tags. The trade-off is that parquet-go's convenient schema mapping is also a compatibility surface. If incoming files contain nonstandard `LIST` or `MAP` layouts, conversion may produce structs rather than the expected native collections. If an application depends on predictable collection types across many writers, that behavior must be checked against the actual files.

Maintenance, licensing, and adoption costs

The repository is not archived, and the recorded last push is on 2026-05-08. The release history lists v1.6.2 on 2021-12-05, v1.6.1 on 2021-09-26, and v1.6.0 on 2020-12-16. The last push indicates repository activity after those releases, but the README does not explain whether every post-release change is covered by a new version or whether consumers should pin a particular commit. The maintenance signal is therefore mixed: recent repository activity exists, while the newest listed release is several years old.

The project uses the Apache-2.0 license. That is a permissive software license, but this review does not provide legal advice. Users still need to review the repository's `LICENSE` file and comply with its notice, license, and patent provisions. The `go.mod` also brings in several dependencies, including Apache Arrow, Apache Thrift, AWS SDK for Go, compression libraries, and `parquet-go-source`. Dependency updates and compatibility checks are part of the operational cost of adopting the module. The `Makefile` exposes the documented test command:

go test --count 1 -v -cover ${PACKAGES}

The Makefile computes `PACKAGES` with `go list ./... | grep -v example`. This command is useful for a repository contributor or integrator, but it does not by itself verify compatibility with a consuming application's schema and files.

Editorial conclusion

Adopt parquet-go when a Go application needs direct struct-based Parquet file I/O and can accept the library's documented type and encoding constraints. Do not choose it solely for claims of speed, because the README states high performance without supplying measurements. Verify first that your required Go types, nested LIST and MAP layouts, compression format, and compatibility with the versions of Parquet files you must read are covered by the repository and its examples. The last push was on 2026-05-08, while the latest listed release is v1.6.2 from 2021-12-05, so maintenance status and dependency compatibility deserve explicit review.

Frequently asked questions

Official sources

1. Issues
2. License: Apache-2.0
3. README
4. Releases
5. xitongsys/parquet-go on GitHub