It is the source code of "Explainable and Safe Reinforcement Learning for Autonomous Air Mobility", which presents a novel deep reinforcement learning (DRL) controller to aid conflict resolution for autonomous free flight. To study the safety under adversarial attacks, we additionally propose an adversarial attack strategy that can impose both safety-oriented and efficiency-oriented attacks.
- Python 3.8
- PyTorch
In order to install requirements, follow:
pip install -r requirements.txt
The definition of environment is in envs
:
-
envs/SimpleATC_env_global.py
is for traditional DQN agent with fixed airways and global perception -
envs/SimpleATC_env_global_v2.py
is for safety-aware DQN(SafeDQN) agent with fixed airways and global perception -
envs/SimpleATC_env_local.py
is for traditional DQN agent with fixed airways and local perception -
envs/SimpleATC_env_local_v2.py
is for safety-aware DQN(SafeDQN) agent with fixed airways and local perception -
envs/SimpleATC_env_local_x.py
is for traditional DQN agent with random airways and local perception -
envs/SimpleATC_env_local_x_v2.py
is for safety-aware DQN(SafeDQN) agent with random airways and local perception
Parameter of the environments can be found in envs/config.py
. And here you can change the related parameters according to your own needs.
By importing different environments in envs
, different models can be trained and evaluated in agents
:
agents/dqn_simple_env
is for traditional DQN agentagents/dqn_simple_env_v2
is for safety-aware DQN(SafeDQN) agent
# take traditional DQN as an example
# For training:
python dqn_simple_env.py --train=True --save_path=" "
# For evaluating:
python dqn_simple_env.py --load_path=" "
You can find the DQN structure in models/dqn_model
.
The adversarial attack methods are in attacks
:
v1
for traditional DQN agentv2
for safety-aware DQN(SafeDQN) agent
Here, we present the demos of four models under 10 routes scenarios without/with adversarial attacks. For safeDQN and safeDQN-X, we only present the results under safety-oriented attacks.
Without Attack | Uniform Attack | Strategically-Timed Attack | |
---|---|---|---|
DQN | |||
DQN-X | |||
safeDQN | |||
safeDQN-X |
Lei Wang 💻 |
Yuankai Wu 💻 |