spark-excel: reading Excel files as Spark DataFrames

What the library does and who maintains it

spark-excel is a library for querying Excel files with Apache Spark, for Spark SQL and DataFrames, and the package description calls it a Spark plugin for reading and writing Excel files.

The readme opens with a request that tells you more about the project than any feature list would. There is a Co-maintainers wanted section, and it says that due to personal and professional constraints, the development of this library has been rather slow. If you find value in this library, please consider stepping up as a co-maintainer, with a link to an issue where you can leave a comment.

The areas it asks for help with are additional features, code improvements and reviews, bug analysis and fixing, documentation improvements, and build and test infrastructure.

That request is worth taking at face value. A library that is still asking for maintainers after years in public use is telling you something real about its future, and the honest thing to do with that information is to weigh it against the alternative of writing the connector yourself.

The project is Scala, licensed Apache 2.0, and published to Maven Central under the dev.mauch group. The last push was on 2026-10-12 and the recent releases include v0.34.0 for Spark 4.1 from 2026-03-29 and v0.32.2 from 2025-10-13, with a v0.32.1-alpha1 that was a test build for shading commons-compress into a thin JAR.

The coordinates are unusual and worth noting before you try to resolve them. The Maven Central badge points at dev.mauch:spark-excel_2.13, so the artifact is under the original author's group rather than the current repository organisation, and the version string carries the Spark version as a prefix. For Scala 2.12 the coordinates are groupId dev.mauch, artifactId spark-excel_2.12, with a version of the form spark-version_0.18.0.

The library requires Spark 3.0 or later, and for the list of Spark versions that are automatically tested the readme points at build.mill and the CI workflow.

Two implementations, and which one to use

Since version 0.14.0, released on 24 August 2021, there are two implementations of spark-excel, and the distinction is the first thing to understand.

The original Spark-Excel uses the Spark data source API 1.0. Spark-Excel V2 uses the data source API V2.0 or later, which supports loading from multiple files, corrupted record handling and some improvements on handling data types.

To use the V2 implementation, you just change your format from .format("dev.mauch.spark.excel") to .format("excel").

So the two are selected by the format string and nothing else, which is a slightly awkward arrangement, since the shorter and more modern name is the generic one and the longer and older name is the project-specific one. The readme is explicit about the mapping, and the changelog is the place to look for the latest features and fixes.

The V2 differences are the practical reason to choose it. Loading from multiple files matters the moment you have more than one workbook, because the older implementation cannot do it at all. Corrupted record handling matters for real data, because spreadsheets in the wild contain broken rows and the failure mode of a connector is to abort the job. And the data type handling improvements cover the Excel-specific cases where a cell looks like one thing and is another.

The interactive shell route is also documented. The package can be added to Spark using the --packages command line option:

$SPARK_HOME/bin/spark-shell --packages dev.mauch:spark-excel_<scala-binary-version>:<spark-version>_<spark-excel-version>

So the coordinates follow the usual Spark convention of artifact, Spark version and library version in one string, and the shell will resolve the package at startup.

The Scala API and its options

There are two ways to read a workbook, and the second one is the convenience form.

The first is the DataFrameReader directly, with the format and the options spelled out. The header option is required. The dataAddress selects the sheet and the range, in the Excel address notation, and defaults to A1. The schema can be supplied explicitly with .schema, in which case it defaults to either the inferred schema or all columns as Strings.

The second form is an implicit that wraps the DataFrameReader returned by spark.read and provides an .excel method which accepts all possible options and provides default values:

import dev.mauch.spark.excel._

val spark: SparkSession = ???
val df = spark.read.excel(
    header = true,  // Required
    dataAddress = "'My Sheet'!B3:C35", // Optional, default: "A1"

The options themselves are where the Excel-specific knowledge lives, and each one has a default that documents its own behaviour.

treatEmptyValuesAsNulls defaults to true. setErrorCellsToFallbackValues defaults to false, and where errors will be converted to null, so that if true, any ERROR cell values such as #N/A will be converted to the zero values of the column's data type. That distinction is worth understanding, because turning an error into null and turning it into a zero are very different things to a downstream aggregation.

usePlainNumberFormat defaults to false, and if true formats the cells without rounding and scientific notations. inferSchema defaults to false, and addColorColumns defaults to false, which is how you get the fill colour of a cell as a column.

The format options are dateFormat, defaulting to yyyy-MM-dd, and timestampFormat, defaulting to yyyy-mm-dd hh:mm:ss[.fffffffff]. Both are passed to the underlying parser, so the default timestamp format accepts nanosecond precision.

The size options are the ones that matter for large workbooks. maxRowsInMemory, defaulting to None, uses a streaming reader which can help with big files, and it will fail if used with xls format files. maxByteArraySize overrides the Apache POI limit on byte array size. tempFileThreshold is the number of bytes at which a zip entry is regarded as too large for holding in memory and the data is put in a temp file instead. And excerptSize, defaulting to 10, is the number of rows to infer schema from when the schema is inferred.

workbookPassword handles encrypted workbooks, and the readme notes it requires unlimited strength JCE for older JVMs.

Selecting a sheet, and reading names dynamically

The sheet selection deserves its own treatment because it has three forms, and the choice between them is not obvious from the API.

The first form is the named address in the dataAddress option, which uses the Excel notation with the sheet name in single quotes and a range, for example 'My Sheet'!B3:C35. This is the form most people use and it works for anything where you know the sheet name in advance.

The second form is an index, for when the sheet name is unavailable:

val df = spark.read.excel(
  header = true,
  dataAddress = "0!B3:C35"
).load("Worktime.xlsx")

So the address is the index followed by an exclamation mark and the range. That is a small convenience, and it is the one to reach for when a workbook has been renamed or generated with a name you do not want to hardcode.

The third form is reading the names dynamically, which is the most robust of the three:

import dev.mauch.spark.excel.WorkbookReader
val sheetNames = WorkbookReader( Map("path" -> "Worktime.xlsx")
                               , spark.sparkContext.hadoopConfiguration
                               ).sheetNames
val df = spark.read.excel(
  header = true,
  dataAddress = sheetNames(0)
)

The WorkbookReader takes a map with a path and the Hadoop configuration from the Spark context, and returns the sheet names. The path is given as a Hadoop-style path rather than a plain filename, which is what lets it work against files in object storage as well as on a local disk.

The distinction matters because a name index is a guess about which sheet you want and a name lookup is not. If a workbook has a cover sheet, a summary and the actual table, then sheetNames(0) may not be the sheet you meant, whereas searching for a sheet by name is unambiguous. The cost of the dynamic approach is an extra read of the workbook to enumerate the sheets, which for a large file is not free.

So there is a sensible progression here. Use the named address when the name is stable, the index when you have checked the order, and the reader when the workbook is generated and you cannot rely on either.

The build tooling and the repository layout

The top level of the repository shows a Scala project with a few choices worth naming.

The build is Mill, with a build.mill file at the root and a mill launcher script alongside it. Mill is a build tool that executes tasks written in Scala, and using it rather than sbt means the build logic is ordinary code you can read and step through, which suits a library where the multi-version Spark matrix is the hard part of the build.

That matrix is what build.mill is for. The readme points at it for the list of Spark versions that are automatically tested, which tells you that the build definition is where the version matrix lives. A library that has to be published for several Spark and Scala combinations has a combinatorial build problem, and keeping it in one executable file rather than in a sprawl of sbt cross-build commands is a defensible approach.

Alongside it are the usual Scala project files. There is a .scalafmt.conf for formatting and a scalastyle-config.xml for the style checks, and a .scala-steward.conf for the dependency update bot. The .git-blame-ignore-revs file is the touch that tells you the project has done a large-scale reformat at some point, because that is the standard way to keep history readable after a tooling change touches every file.

There is also a private-key.pem.enc in the root. An encrypted private key checked into the repository means the build can sign published artefacts, and the key itself is encrypted rather than stored in plain text. The presence of the file is the reason to trust that the project publishes signed artefacts, though the readme does not describe the signing setup in detail.

The remaining top level entries are the docs/ directory, src/, a CHANGELOG.md, CONTRIBUTING.md, ISSUE_TEMPLATE.md and a LICENSE. The changelog matters here because the readme explicitly points at it for the latest features and fixes, and with the two implementations coexisting since 2021 it is the place to work out which behaviour you are actually getting.

The repository is maintained under the nightscape organisation but published under the dev.mauch Maven group, which is a small inconsistency that will cost you a few minutes if you go looking for the artefact by organisation name.

Editorial conclusion

Adopt spark-excel if you need Excel workbooks as Spark DataFrames and have found that the built-in CSV path does not handle multi-sheet workbooks, cell formatting or password-protected files. Do not adopt it without reading the maintenance note, because the project explicitly asks for co-maintainers and says development has been slow. Before you commit, use the V2 implementation rather than the original one, since the format name differs only slightly and the older path lacks multi-file support.

Frequently asked questions

Official sources

1. Issues
2. License: Apache-2.0
3. nightscape/spark-excel on GitHub
4. README
5. Releases