aerial-autonomy-stack: simulating and deploying perception based drone swarms

One codebase across three flight stacks and two deployment targets

Aerial autonomy stack, abbreviated AAS in the README, is described as a batteries included, flight-proven, open software and the simplest and fastest way to do three things: develop multi-drone autonomy with PX4, ArduPilot, Betaflight and ROS2; simulate faster-than-real-time perception and control with YOLO and 3D LiDAR; and deploy in real drones with JetPack, DeepStream and NVIDIA Orin.

Those three verbs define the project, and the important word in them is the same. The autonomy logic is written once, against a ROS 2 interface, and it runs either against a simulated vehicle or against a real one. The simulation is not a separate codebase that pretends to be the real thing; it is the same autonomy code pointed at a different vehicle.

The supporting cast is what makes that possible. The autopilot side is abstracted behind a ROS 2 action-based interface, reachable via XRCE-DDS, MAVROS or MSP depending on which flight stack you are using. The perception side is YOLO with ONNX GPU runtimes for detection and LiDAR odometry built on KISS-ICP. The simulation side is Gazebo with its wind effects and the asv_wave_sim waves plugin, plus a steppable Gymnasium environment that supports faster-than-real-time and multi-instance simulation.

The repository layout mirrors that division into four top level directories, aas-gym for the reinforcement learning environment, aircraft for the vehicle side, ground for the ground control side, and simulation, plus tools_and_docs for the scripts and documentation.

The primary language is C++, the licence is MIT, the homepage is an arXiv paper, and the release tags are named after what changed: v3.0.0 for the Betaflight release on 2026-10-18, v2.0.0 for the JetPack 7 release in August 2026, and v1.6.0 for the Advanced SITL release in July 2026. The last push was on 2026-10-22.

Installation is a 45 minute container build, not a package install

The installation section is short and the build time is stated plainly, which is the most useful thing about it.

AAS is developed with Ubuntu 24.04 and nvidia-driver-615-open on an i7-11 with an RTX 3060, and the README points to REQUIREMENTS_UBUNTU.md, or REQUIREMENTS_WSL.md for Windows 11, to install the requirements.

sudo apt update && sudo apt install -y git xterm xfonts-base wget unzip

git clone https://github.com/JacopoPan/aerial-autonomy-stack.git
cd aerial-autonomy-stack/tools_and_docs/

./tests/check_requirements.sh                         # AAS requires nvidia-driver, docker, and nvidia-container-toolkit
./sim_build.sh                                        # The 1st build takes ~45' with good internet (`Ctrl + c` and restart if needed, cached stages will be preserved)

Two things in that block deserve attention. The check_requirements.sh script verifies the three things the stack cannot work without, an nvidia-driver, docker and the nvidia-container-toolkit, and running it first turns a confusing failure later into an obvious one now. And the comment on the build says the first build takes about 45 minutes with good internet, with a note that you can press Ctrl and c and restart if needed because cached stages will be preserved, which tells you the build is a multi stage image build and that an interrupted run is recoverable rather than a restart from zero.

The environment is a container, and the containers are NVIDIA based. The simulation is dockerized on Ubuntu with CUDA and cuDNN, and the deployment is dockerized on Ubuntu with CUDA, cuDNN, TensorRT and DeepStream. Windows 11 is supported via WSL.

There is also a pre-built image route, and it is worth knowing about if the 45 minute build is the wrong trade for a first look.

# ghcr.io images are re-built from `main` every Friday night
for name in aircraft ground simulation; do
  docker pull ghcr.io/jacopopan/${name}-image:latest
  docker tag ghcr.io/jacopopan/${name}-image:latest ${name}-image:latest
done

Three images, one for each of the aircraft, ground and simulation sides, rebuilt weekly from main. The trade is obvious: you get the current main rather than the commit you read about, so a reproducibility-minded team should build from the clone instead.

Running a simulation, and the options that shape it

A simulation is started from the tools_and_docs directory with a single script, and the README documents the options in the script itself.

cd aerial-autonomy-stack/tools_and_docs/

NUM_QUADS=1 NUM_VTOLS=1 WORLD=swiss_town RTF=3 PLOT=true ./sim_run.sh    # Start a simulation, check the script for more options (note: ArduPilot SITL checks take ~30s of simulated time before being ready to arm)

The variables cover the autopilot choice with AUTOPILOT=px4, ardupilot or betaflight, the sensors with HEADLESS, CAMERA and LIDAR, and the PX4 ROS 2 library flag, the vehicle counts with NUM_QUADS, NUM_VTOLS and NUM_TAILS, the odometry source with ODOM=none, openvins, fastlio, superodom or mimosa, the world with WORLD=impalpable_greyness, apple_orchard, crematoria, shibuya_crossing, swiss_town or waterworld, the real-time factor with RTF, the parallel instance number with INSTANCE, and the plotting and recording flags.

Two of these are worth understanding properly. RTF is the real-time factor, where 1 is real time and 0 means as fast as possible, and the comment notes that PX4_ROS2_LIB=true or AUTOPILOT=betaflight set RTF=1. So the faster-than-real-time claim has a boundary: some configurations are pinned to real time. And the odometry options note that all except none require BUILD_ADVANCED_ODOM=true, which is a build flag, so choosing a different odometry source is a rebuild rather than a config change.

The ArduPilot caveat about 30 seconds of simulated time before being ready to arm is the kind of detail that saves a lot of confusion, since with a real-time factor of 3 that is ten seconds of wall clock and looks like a hang.

The stack also supports Jetson-in-the-loop simulation, described as multi-HITL simulation to test NVIDIA and ARM based on-board compute, and uses a dual network to separate simulated sensors on SIM_SUBNET from inter-vehicle comms on AIR_SUBNET, with a Zenoh based inter-vehicle ROS 2 bridge. That separation is what makes a swarmed simulation tractable, since the sensor traffic of one vehicle does not pollute the communication channel of the others.

Three ways to make the drones fly, from formation to behaviour tree

There are three ways to autonomously fly the drones, plus a USB gamepad for manual flight and QGroundControl or the Betaflight app for operator supervision.

The first is a synchronized formation flown from the Ground terminal, using a node called dtc_controller_node.

ros2 run drone_traffic_controller dtc_controller --ros-args -p use_sim_time:=true

The second flies a single vehicle's own behaviour tree mission from that vehicle's terminal, with the mission file being a YAML such as yalla.yaml.

ros2 run mission mission --conops yalla.yaml --ros-args -r __ns:=/Drone$DRONE_ID -p use_sim_time:=true

The third is the ROS 2 actions route, and this is the interface a serious project would build on, because an action is cancellable and has a defined goal and result.

cancellable_action "ros2 action send_goal /Drone${DRONE_ID}/takeoff_action \
    autopilot_interface_msgs/action/Takeoff '{takeoff_altitude: 30.0}'"

cancellable_action "ros2 action send_goal /Drone${DRONE_ID}/offboard_action \
    autopilot_interface_msgs/action/Offboard \
    '{controller_name: att-test, max_duration_sec: 10.0}'"
# Add or re-implement offboard controllers in `px4_offboard.cpp`, `ardupilot_guided.cpp`, `betaflight_rc_override.cpp`

Underneath, the three action types are backed by three separate controller implementations, one per flight stack, and the last line of the block is the important instruction: you add or re-implement offboard controllers in those three source files. That is where the abstraction is thin.

Each stack brings its own semantics. PX4 has the Offboard interface, with CTBR and VehicleRatesSetpoint for agile GNSS-denied flight, and there is support for the PX4 ROS 2 Interface Library for external flight modes. ArduPilot has the Guided interface with setpoint_velocity and setpoint_accel references. Betaflight has the MSP Override interface, with RC sticks in ACRO, ANGLE or ALTHOLD plus POS HOLD.

So the action API is uniform and the semantics underneath are not, which is the central design tension in the project and the thing a user has to understand before building on it.

Flight logs, and the analysis tools that come with them

The logging story is one of the more complete parts of the project, and it is worth knowing about before you fly rather than after you need it.

Logs analysis is supported with flight_review for .ulg files, which is the PX4 native format, MAVExplorer for .bin files, and PlotJuggler for MCAP rosbag files. Those are three different toolchains for three different autopilot log formats, which tells you the project is genuinely supporting all three flight stacks rather than treating Betaflight as an afterthought.

The recording is done from the simulation side, and the flags in the sim_run.sh options are PLOT and RECORD_ROSBAG, with the comment that plotting requires pymavlink, pyulog and pymap3d. So the plotting path is a Python stack reading the recorded logs, and RECORD_ROSBAG is separate, producing the rosbag that PlotJuggler consumes. The separation between the two is worth noting: the rosbag is the ROS side of the story and the pymavlink and pyulog path is the autopilot side.

The three release tags also tell you the order in which the project was extended, which is a rough guide to what is most settled. The Advanced SITL release came first in July 2026, then the JetPack 7 release in August, then the Betaflight release in October. SITL is the software in the loop simulation, and the sequence suggests the simulation was the foundation, the deployment on new Jetson hardware the next step, and the third flight stack the most recent addition.

The documentation set is split across the tools_and_docs directory, with a BOM.md giving an example bill of parts and a RATIONALE.md giving the motivation. A bill of parts in a project like this is unusually valuable, because the hardware side of a perception drone stack, what Jetson, what camera, what LiDAR, is the part that determines whether your autonomy code transfers from simulation to the field.

What this stack does not do for you

Two boundaries are worth naming before anyone plans a flight with it.

The first is the environment. The whole perception and deployment path is built on NVIDIA hardware, CUDA, cuDNN, TensorRT and DeepStream, and the development machine is described as an i7-11 with an RTX 3060. The simulation can run faster than real time precisely because the heavy perception work is on a GPU, which is also why a machine without a suitable NVIDIA GPU is not a supported configuration. A team whose perception stack runs on something other than NVIDIA hardware gets the flight interfaces and the simulation world, and then has to write the detection and odometry parts themselves.

The second is what the stack is for. The README describes development, simulation and deployment, and the notable omission is any claim about flight certification, airworthiness or operational approval. That is entirely reasonable for a research harness, and it is also the boundary: the simulation tells you the autonomy code works against a simulated world with Gazebo's wind and wave plugins, and it tells you nothing about whether the aircraft will fly safely in the air, where the environment is not a plugin.

The third point is about scope of the abstraction. The action API is uniform and the semantics are not, and the fix is a source change in three separate controller implementations. For a project doing research across all three stacks, that is a reasonable price and the README says so directly. For a project that has picked one autopilot and wants a clean library, the uniformity is a distraction and the three-file maintenance cost is real.

The pragmatic comparison is with building the pieces separately: a Gazebo world with PX4 on its own, a YOLO pipeline on its own, a ROS 2 interface on its own, and a Jetson deployment script on its own. That path gives you less integration and more freedom, and it means each piece can be replaced. The path this project takes gives you one container build, one set of entry points, and the same code on both sides of the simulation boundary, at the cost of accepting the environment it specifies and the three controller files it asks you to edit.

Editorial conclusion

Aerial autonomy stack fits a research team that wants one codebase spanning Gazebo simulation and real Jetson hardware, with the same ROS 2 interface either side of the flight, and it does not fit a team that needs to fly commercially or that has no NVIDIA hardware in the loop, because the perception and deployment path is built around Jetson and CUDA. Before trying to run it, read REQUIREMENTS_UBUNTU.md, or REQUIREMENTS_WSL.md on Windows 11, and expect the first build to take about 45 minutes with a good connection, since the sim_build.sh script builds container images rather than installing a package.

Frequently asked questions

Official sources

1. JacopoPan/aerial-autonomy-stack on GitHub
2. License: MIT
3. Project website
4. README
5. Releases