Apache Tika 4.x: Markdown-first extraction with crash isolation for agent pipelines

What Tika 4.x actually changes

Apache Tika has long been the default Java answer to the question "what is in this file?". The 4.x line, as described in the README, makes three deliberate breaks with the past. First, Markdown becomes the default output format instead of plain text. That is a direct response to LLM and RAG pipelines, which prefer structured text with headings and lists. Second, parsing runs in crash-isolated forked processes. A hostile or malformed document can kill a fork, not the host service. Third, the project now ships vision-language-model parsers for Claude, Gemini, and OpenAI, aimed at documents that OCR cannot read. These are not incremental tweaks; they reposition Tika from a metadata library into a component for agent workflows. The README even includes ready-to-use agent skills in the .skills/ directory, which are standalone and can be copied into any agent's skill directory without requiring this repository.

How the extraction mechanism works

The core flow remains the same as earlier Tika versions: a detector identifies the file type, then a parser extracts text and metadata. What changed in 4.x is the default serialization and the process boundary. The command-line examples show three modes. The default invocation, java -jar tika-app-<version>.jar document.pdf, emits Markdown. The --text flag returns plain text for those who want the old behavior. The -J flag produces structured JSON that includes metadata and content for the document and for anything embedded in it, such as images or attachments inside a PDF. That recursive extraction is what the README calls structured recursive extraction, and it is the mechanism agent pipelines are expected to consume. The forked-process isolation is not a plugin; it is part of the parsing architecture, so a crash in a parser does not take down the JVM that hosts the service. The vision-language-model parsers sit alongside the traditional ones, but the README does not describe how they are invoked or configured, only that they exist for documents OCR cannot read.

Getting it running: commands and build options

The quickest path is the pre-built tika-app zip from the download page. The README warns that the zip has no top-level directory, so you must unzip it into its own folder and run from inside it. The jar is a thin launcher; it loads parsers from the adjacent lib/ directory and fails with NoClassDefFoundError if run alone. Three commands cover the basics: java -jar tika-app-<version>.jar document.pdf for Markdown, the same with --text for plain text, and with -J for structured JSON. For Java projects, the Maven dependency is org.apache.tika:tika-parsers-standard-package with type pom. The README also shows a BOM artifact, tika-bom, to align module versions and avoid convergence errors. Building from source requires Java 17 and Maven, with the included mvnw wrapper. A full build is ./mvnw clean install, which produces a runnable tika-app. For faster iterations, the README suggests -Pfast to skip tests and checkstyle, -T1C for parallel builds, and mvnd for a warm JVM. If the ossindex-maven-plugin fails the build because a dependency has a known vulnerability, you can skip it with -Dossindex.skip. Reproducible builds are supported: building the same source with the same JDK produces byte-for-byte identical artifacts, and you can verify with ./mvnw artifact:check-buildplan.

Limitations and failure modes

The most obvious limitation is the end-of-life schedule. Tika 2.x and Java 8 support ended in April 2025, so anyone on those lines must upgrade to 3.x or 4.x, and the migration to 4.x is not trivial. The README lists breaking changes: Java 17 is required, TikaInputStream appears in the Parser and Detector SPI, configuration moves from tika-config.xml to JSON, metadata keys become namespaced, and Markdown becomes the default output. That is a lot of surface area for a minor version bump. Another failure mode is the thin launcher jar. If you grab the jar alone, it will not work; you need the whole zip layout. The README is explicit about this, but it is easy to miss and results in a confusing NoClassDefFoundError. The vision-language-model parsers are a notable gap in the documentation: the README mentions them but gives no example of how to configure API keys, which models are supported, or what the output looks like. That makes it hard to evaluate whether they are production-ready or experimental. Finally, the forked-process isolation is a design choice that trades memory and startup latency for safety. Each parse may spawn a new JVM, which is fine for batch jobs but could be a bottleneck for low-latency, high-throughput services.

Alternatives and how they differ

The closest alternative for Java developers is Apache POI, which Tika itself uses under the hood for Office formats. POI is a lower-level library: you must know the file type in advance and call the specific API for XLSX, DOCX, or PPTX. Tika abstracts that away with detection and a uniform parseToString method. If you need only Office files and want fine-grained control over cell values or styles, POI is the better fit. For Python-centric pipelines, libraries like textract or pypdf offer extraction for a narrower set of formats and do not provide the same metadata depth or process isolation. The key difference is that Tika is a server-capable, multi-format toolkit with a Java SPI, while those are single-purpose parsers. Tika also offers tika-server-standard as a separate module, which the README mentions as a build target, suggesting a REST service option, though the README does not document its endpoints. The choice comes down to whether you want a single entry point for a thousand formats or a specialized parser for a few.

Maintenance and upgrade cost

Tika is an Apache Software Foundation project, which means a governance structure and a public roadmap. The README points to a Tika Roadmap wiki page for the support schedule of each line, and the EOL announcement for 2.x shows that the project is willing to cut off old versions. The migration to 4.x is the main cost: code changes, configuration format changes, and a new default output. The README provides a migration guide, but it is truncated in the material, so the exact steps are not visible here. The build system is Maven with a wrapper, and the project supports reproducible builds, which lowers the risk of supply-chain surprises. The ossindex-maven-plugin is part of the build and can fail on known vulnerabilities, which is a maintenance burden but also a security feature. The license is Apache-2.0, which is permissive for commercial use, but you should check the dependencies of the parsers you enable, since some file-type parsers may pull in third-party libraries with different licenses. The README does not enumerate those dependencies, so that is a verification step before adoption.

Editorial conclusion

Adopt Tika 4.x if you need a single library that extracts text and metadata from over a thousand file types and you want Markdown-shaped output for LLM or RAG pipelines. Skip it if you still run Java 8 or Tika 2.x, because those lines reached EOL in April 2025. Before committing, verify your document corpus against the new JSON configuration and namespaced metadata keys, and test the forked-process mode in your deployment, since the default behavior changed from 3.x and the migration guide is the authoritative source for required code changes.

Official sources

1. Official documentation
2. Official README
3. Project repository