UAV-DDPG: A Paper's Simulation Code for Offloading Tasks to a Flying Edge Server

What the UAV offloading problem actually is

A user equipment has a computation task and two places to run it: on its own CPU, or on a UAV that is flying nearby and carrying computing resources. The paper's model splits the task, so a portion goes over the air to the UAV and the rest stays local. The objective is not total energy or average latency. It is the maximum processing delay across all user equipments, and the paper states that this maximum is minimized by jointly optimizing user scheduling, the task offloading ratio, the UAV flight angle and the UAV flight speed. The delay for one UE is computed in com_delay() as the larger of two quantities: transmission time plus edge computation time, and local computation time. The reward returned by the environment is simply the negative of that maximum delay, so a reinforcement learning agent that maximizes reward is minimizing the worst-case delay. The intended audience is narrow. This is code for researchers who work on UAV-assisted mobile edge computing, deep reinforcement learning for resource allocation, or both, and who want a runnable environment in which the offloading decision, the flight decision and the scheduling decision are coupled. Engineers looking for a scheduling component to embed in a real network will find the abstractions too coarse.

The environment: a 100m cube, four UEs and 40 slots

UAV_env.py defines the whole world. The flight volume is 100m by 100m by 100m, set by height, ground_length and ground_width all equal to 100. The channel bandwidth B is 1 MHz and the nominal flight speed is 50 m/s. There are M = 4 user equipments, and an episode is divided into slot_num = T / (t_fly + t_com) slots, which the README puts at 40 per episode. The state vector has dimension 4 + M*4 and is laid out explicitly in the README: battery remaining, UAV x and y, remaining total task size, then for each UE its x and y position, its task size, and a block flag that records whether the link is line-of-sight or non-line-of-sight. That block flag matters because the channel gain follows free-space path loss with separate LOS and NLOS noise levels, so the same distance can produce a different transmission rate depending on the flag. The action vector has four continuous components in [-1, 1], rescaled to [0, 1] and then mapped to their physical ranges: the target UE index over [0, M-1], the flight angle over [0, 2*pi], the flight distance over [0, flight_speed * t_fly], and the offloading ratio over [0, 1]. The step() method decodes the action, computes flight energy, moves the UAV, calls com_delay(), updates battery and task state, and returns a tuple that includes a step_redo flag alongside the usual next state, reward and terminal signal. That fifth element is unusual and worth noting: the environment can ask for a step to be redone, which is a mechanism the training loop has to respect rather than a detail you can ignore.

The DDPG agent and one hyperparameter that stands out

ddpg_algo.py holds the agent. The actor is a four-layer network: state into 400 units with relu6, then 300 with relu6, then 10 with relu, then a 4-dimensional tanh output scaled by action_bound. The critic embeds state and action separately into 400 units each, sums them with a bias, and passes the result through 300 relu6, 10 relu and a linear output that produces the Q-value. Experience replay uses a buffer of MEMORY_CAPACITY = 10000 transitions and a BATCH_SIZE of 64. The learning rates are LR_A = 0.001 for the actor and LR_C = 0.002 for the critic, and the target networks are updated softly with TAU = 0.01. The discount factor is GAMMA = 0.001. That last value deserves a second look. DDPG implementations commonly use a discount near 0.99, and a value of 0.001 means the agent barely values future reward, which in a problem whose objective is a delay accumulated over a 40-slot episode is a strong design choice. The README presents it as the configured value without explaining it, so if you plan to build on this code, treat the discount factor as the first thing to interrogate rather than the last. The training loop adds Gaussian exploration noise N(0, var) to the chosen action, executes a step, stores the transition, and calls learn(), which soft-updates the targets and then trains actor and critic on a sampled batch.

Baselines and ablations are in the tree, not in a separate repo

The repository layout puts the comparisons next to the main algorithm. DQN/ contains its own UAV_env.py with a discrete action space, plus dqn_algo.py and a state_normalization.py. Actor Critic/ contains ac_algo.py, described as actor-critic with a continuous action space. Edge_only/ and Local_only/ are the two degenerate policies: offload everything to the UAV, or offload nothing. Two ablation directories sit under DDPG/: DDPG_without_behavior_noise and DDPG_without_state_normalization. The ablation on state normalization is the more interesting of the two, because state_normalization.py exists as a first-class module whose stated job is to scale state values to [0, 1], and the state vector mixes quantities with very different magnitudes: a battery level, coordinates in metres, task sizes, and a binary block flag. Removing that scaling is a plausible way to show it matters. Note the directory name Actor Critc, which is misspelled in the repository. That is harmless for imports if the scripts use relative paths, but it will break any tooling that reconstructs module paths from the directory name, and it is the kind of detail that tells you this tree was assembled for a paper rather than for distribution.

Getting it running: three dependencies and no setup file

The required software list is short: TensorFlow 1.X, NumPy and Matplotlib. There is no requirements.txt, no setup.py, no conda environment file and no Dockerfile in the material provided, so dependency resolution is on you. The practical consequence is that the first obstacle is not the algorithm but the interpreter. TensorFlow 1.X is long past its mainstream support window, and the README does not say which 1.x minor version was used, so a fresh install of the latest 1.x is a guess rather than a reproduction. The homepage at fangvv.github.io/UAV-DDPG is listed as the project page, and the paper itself is linked from the README to a Springer article in Wireless Networks, which is where the experimental setup is described. There are no releases in the repository metadata, so there is no tagged version to pin. If you want a stable reference point, the only one available is a commit hash on the main branch. Running the code means running the scripts in DDPG/, DQN/, Actor Critic/, Edge_only/ and Local_only/ individually; the README does not document a single entry point that produces the paper's figures.

Where this code will resist you

The environment is a simulator written for one paper's system model, and it is not parameterized for substitution. The number of UEs is a class attribute M = 4, the area is fixed at 100m on each side, the bandwidth is fixed at 1 MHz, and the channel model is free-space path loss with LOS and NLOS noise levels baked into the gain calculation. Changing the radio model means editing com_delay() and the state construction together, because the block flag is both a channel input and a state feature. The same coupling applies to energy: flight energy is computed inside step(), so an energy model that depends on payload or wind has nowhere to attach without restructuring the method. The second limitation is the TensorFlow 1.X dependency, which is a real constraint rather than a stylistic preference. Anyone whose environment has moved to TensorFlow 2.x or PyTorch faces a port, and the actor and critic builders use scoped variable names and a soft target update pattern that will not translate line for line. Third, the README's state and action tables are the most useful documentation in the repository, but the training section is truncated in the material available, so the exact episode count, noise schedule and evaluation procedure used for the published curves are not recoverable from what is shown here.

The licence question and what it means for reuse

The repository metadata shows no licence. That is not a formality for a project like this one. Without a licence file, the default copyright position applies, and the code is published for reading and for reproducing the paper's results rather than for incorporation into another project. The README does not state terms, and the homepage does not either, based on the material provided. If you intend to reuse any part of UAV_env.py or ddpg_algo.py in your own work, the absence of a licence is the first thing to resolve, and that is a question for the author or for whoever handles licensing at your institution. I am not giving legal advice here; the point is simply that you cannot infer permission from public availability. The maintenance picture is similar. The repository is not archived and the last push is recent, but there are no releases, no changelog and no issue templates in the metadata, so there is no signal about how fixes are handled or whether the TensorFlow 1.X dependency will ever be lifted.

How it compares with a general-purpose DRL toolkit

The obvious alternative for someone who wants DDPG is not another offloading repository but a general reinforcement learning library, for example Stable-Baselines3. The difference in approach is structural. A library like Stable-Baselines3 implements DDPG and its variants as algorithms that consume an environment conforming to the Gym interface, and it maintains its own training loop, replay buffer, noise process and logging. Here the algorithm and the environment are co-designed: the action decoding, the step_redo flag, the reward of negative maximum delay and the state normalization are all specific to this MEC model, and the agent is written to match. That coupling is the reason this repository is useful for the paper's problem and the reason it is awkward everywhere else. If your environment already conforms to a standard interface, a general library gives you tested implementations, checkpointing and logging for free. If your problem is exactly this UAV offloading problem, the library gives you none of the domain logic and you would be writing com_delay() and the state layout yourself anyway. The trade is domain fit against ecosystem support, and this repository sits firmly on the domain-fit side.

Editorial conclusion

Adopt this repository if you are reproducing the Wireless Networks paper or want a compact DDPG reference implementation with a non-trivial environment already wired in: the 4 + M*4 state vector, the four-dimensional continuous action, and the reward of negative maximum delay are all in UAV_env.py. Do not adopt it as a production offloading controller, and do not expect plug-and-play support for your own radio model, because the channel and energy models are hard-coded in the environment class. Before you commit, verify three things: that a TensorFlow 1.x interpreter is still available to you, that the repository actually carries a licence file (the metadata shows none), and that the reward scaling, GAMMA = 0.001 in particular, is what you want, since that value differs from the usual DDPG default.

Official sources

1. fangvv/UAV-DDPG on GitHub
2. Issues
3. Project website
4. README