QUAST: the assembly quality report for genome and metagenome comparisons

What QUAST is, and the four tools in the package

QUAST stands for QUality ASsessment Tool, and it evaluates genome and metagenome assemblies by computing various metrics. The package is not one program but a small family. The general QUAST tool handles genome assemblies, MetaQUAST is the extension for metagenomic datasets, QUAST-LG is the extension for large genomes such as mammalians, and Icarus is the interactive visualizer for these tools. The homepage is quast.sf.net and the online manual sits at quast.sf.net/manual.html, which is where the much more detailed description of installation, basic usage and output parsing lives. The project is maintained by the Gurevich Lab at the Helmholtz Institute for Pharmaceutical Research Saarland, and the licence is GPL v2 with the copyright and citation instructions in LICENSE.txt.

One design decision shapes how you use it. The QUAST package works both with and without reference genomes, and the readme is candid that it is much more informative if at least a close reference genome is provided along with the assemblies. The tool accepts multiple assemblies, which is what makes it suitable for comparison, so the normal workflow is not one assembler and one score but several candidate assemblies and a table of how they differ.

The top level of the repository reflects the family. There is quast.py for the general tool, metaquast.py and quast-lg.py for the two extensions, and icarus.py for the visualizer, plus quast_libs/ for the shared code, external_tools/ for the third-party programs it drives, and a tc_tests/ directory for the test suite. There are also two old-style installation scripts, install.sh and install_full.sh, and a Dockerfile for container use.

System requirements, and the optional pieces

Linux 64-bit and macOS are supported, and the main pipeline needs Python 3 at version 3.3 or higher, Perl 5.6.0 or higher, GCC 4.7 or higher, GNU make and ar, and the zlib development files. Most of those are usually preinstalled on Linux, and the readme notes that macOS requires installing the Command Line Tools for Xcode to make them available.

The optional pieces matter for specific flags rather than for the whole tool. The Time::HiRes perl module is needed for GeneMark-ES, which is what you use with the gene-finding and eukaryote options. Java 1.8 or later is needed for GRIDSS, which is the structural variation detection, and so is R, for the same feature. So a run without structural variation detection needs neither, and adding it later means installing a JVM and an R installation rather than just a Python package.

There is one more group of requirements for the plots. QUAST draws plots in two formats, HTML and PDF, so if you need the PDF versions you need Matplotlib installed, version 1.1 or higher, and the tool is fully tested with Matplotlib 1.3.1. On Ubuntu the readme gives the command that covers the plotting dependencies.

sudo apt-get update && sudo apt-get install -y pkg-config libfreetype6-dev libpng-dev python3-matplotlib

Installing, and the choice between two sizes

The unusual thing about QUAST installation is that it is optional. QUAST automatically compiles all its sub-parts when needed, on the first use, so installation is not required if you are happy to run it from the checkout. If you want to precompile everything and add quast.py to your PATH, there are two routes, and the difference between them is what gets included.

The basic installation is about 120 MB.

./setup.py install

The full installation is about 540 MB and adds two things, the tools for structural variation detection based on read pairs, which is used for more precise misassembly detection, and the tools and data for reference genome detection in metagenomic datasets.

./setup.py install_full

The default installation location is /usr/local/bin/ for the executable scripts and /usr/local/lib/ for the Python modules and auxiliary files. If you hit a permission error during installation, the readme suggests running setup.py with sudo, or creating a virtual Python environment and installing into that instead, and there are also the old-style installation scripts, install.sh and install_full.sh, which build the QUAST package in place. The project is also on BioConda, on PyPI and on SourceForge, and the readme points at quast.sf.net/install.html for the full set of installation methods for the latest stable release, with the GitHub repository being the development version.

Running it, and the flags that shape a run

The usage example in the readme passes two assemblies, a reference, a gene file and a pair of read files, and writes its output to a directory you name.

./quast.py test_data/contigs_1.fasta \
           test_data/contigs_2.fasta \
        -r test_data/reference.fasta.gz \
        -g test_data/genes.txt \
        -1 test_data/reads1.fastq.gz -2 test_data/reads2.fastq.gz \
        -o quast_test_output

The -r flag is the reference genome, and the readme already told you that supplying one makes the run much more informative. The -g flag points at the gene file, which is what the gene prediction metrics are computed against. The -1 and -2 flags are the two ends of the read pair, and passing them is what enables the read-based metrics and the reads_stats output. The -o flag names the output directory, and the output directory becomes the first thing you look at, because the report and the visualizer are written into it.

What comes out, and where to look

The output directory holds several representations of the same results, and they exist for different audiences. report.txt is the summary table, report.tsv is the tab-separated version for parsing or for spreadsheets, report.tex is the LaTeX version, report.pdf includes all the tables and plots for some statistics, and report.html puts everything into a single interactive file. icarus.html is the Icarus main menu with links to the interactive viewers.

Three subdirectories appear conditionally, and the condition is worth noting because it tells you which inputs did something. contigs_reports/ appears only if a reference genome is provided, and holds misassemblies_report, the detailed report on misassemblies, and unaligned_report, the detailed report on unaligned and partially unaligned contigs. k_mer_stats/ appears only if the k-mer-stats option is specified, holding kmers_report. And reads_stats/ appears only if reads are provided, holding reads_report with the mapped reads statistics.

So the shape of the output tells you what the run was configured to measure. A directory with no contigs_reports is a run without a reference, which is exactly the less informative mode the readme warns about.

The metrics, split by what they need

The metrics are listed in two groups, and the grouping is by the input required rather than by what the numbers mean.

The contig-only metrics need nothing but the assembly. You get the number of large contigs, meaning those longer than 500 bp, and their total length, the length of the largest contig, and N50, defined as the length of a contig such that all the contigs of at least the same length together cover at least 50 percent of the assembly. You also get the number of predicted genes, discovered either by GeneMark.hmm for prokaryotes, GeneMark-ES or GlimmerHMM for eukaryotes, or MetaGeneMark for metagenomes.

With a reference genome the list grows. You get the numbers of misassemblies of different kinds, which are inversions, relocations, translocations, interspecies translocations in MetaQUAST only, and local misassemblies. You get the number and total length of unaligned contigs, the numbers of mismatches and indels both over the whole assembly and per 100 kb, and the genome fraction, the assembled part of the reference. The duplication ratio is the total number of aligned bases in the assembly divided by the total number of those in the reference, and the readme explains that if the assembly contains many contigs that cover the same regions its duplication ratio will significantly exceed 1, for reasons including overestimating repeat multiplicities and overlaps between contigs.

What the tool is built on, and the container route

The Dockerfile shows what a working QUAST environment needs to contain, and it is a useful list even if you are not using Docker. The base is ubuntu:16.04, and the system packages installed are curl, wget, g++, make, libboost-all-dev, git, tar, gzip, bzip2 and build-essential, along with python2.7-dev, python-setuptools, python-pip, python-virtualenv, zlib1g-dev, default-jdk and perl, and libstdc++6 is upgraded at the end of the same layer. Matplotlib dependencies come next as a separate layer with pkg-config, libfreetype6-dev, libpng-dev and python-matplotlib, and the QUAST installation itself is the final stage, where setuptools and pip are upgraded, the source is copied in, and the package is installed in develop mode.

The setup.py file shows the same thing from the code side. It checks the Python version through qconfig, sets up a console logger, and then imports the compile helpers for the external programs it needs, so glimmer, augustus and busco databases, the BLAST binaries and database, the aligner, the reads analyzer tools, BWA, bedtools and GRIDSS. The imports are the clearest statement in the repository of what QUAST actually is, because it is a driver for a whole collection of bioinformatics tools rather than a self-contained assembler.

There is also a dnanexus/ directory at the top level, and other_scripts/, a docs_for_devs/ folder and a manual.html file, so the project ships its own documentation rather than pointing only at the website.

Where QUAST is the wrong tool

Two boundaries are clear. The first is the reference genome. The package works without one, and the readme says it is much more informative if at least a close reference genome is provided, so for a genuinely novel genome, where you have no close reference at all, the metrics you get are the contig-only ones and the tool tells you less. That is a limitation of the question rather than a defect of the software, but it means the tool answers different questions depending on what you bring to it.

The second is the weight of the install. A full installation is about 540 MB and pulls in a JVM, an R installation and a pile of bioinformatics binaries that have to be compiled or fetched, and the old-style scripts build the package in place rather than into a clean environment. On a machine where you do not control the system Python, that is a real friction point, and the readme's own advice is to create a virtual environment and install into that.

A third, smaller point is the platform list. Linux 64-bit and macOS are supported, and nothing else is mentioned, so Windows is not a target. For a tool whose normal home is a Linux cluster or a workstation running Linux, that is not a limitation in practice, but it does mean the container route or a remote machine is the answer rather than a native install.

Editorial conclusion

QUAST fits a bioinformatics team comparing several candidate assemblies of the same genome, since it accepts multiple assemblies at once and produces a report format that is easy to diff between runs, and it fits anyone who wants the standard contig and reference-based metrics computed the same way every time. It does not fit someone working on a genome with no close reference, because the readme says plainly that the tool is much more informative when a reference is provided, and the reference-based metrics are the interesting ones. It also does not fit a Windows user, since only Linux 64-bit and macOS are supported. Before you install, decide between the basic and full builds, since the full one is more than four times the size and adds the read-pair structural variation detection and the metagenomic reference detection data, and set up a virtual environment if you do not control the system Python.

Frequently asked questions

Official sources

1. ablab/quast on GitHub
2. Issues
3. Project website
4. README
5. Releases