Skip to content

[T-ITS] Sim-to-real goal-oriented mapless autonomous navigation (DRL navigation).

License

Notifications You must be signed in to change notification settings

OscarHuangWind/DRL-Transformer-SimtoReal-Navigation

Folders and files

NameName
Last commit message
Last commit date

Latest commit

Β 

History

29 Commits
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 

Repository files navigation

GoT-GTRL

πŸ“ƒ Goal-guided Transformer-enabled Reinforcement Learning for Efficient Autonomous Navigation

πŸ’« A goal-driven mapless end-to-end autonomous navigation of unmanned grounded vehicle (UGV) realized through Transformer-enabled deep reinforcement learning (DRL) algorithm.

πŸš™ A car-like mobile robot learns to autonomously navigate to a random goal position only through raw RGB images from one Fisheye camera and goal information in polar coordination system.

πŸ”§ Realized in ROS Gazebo simulator with Ubuntu 20.04, ROS noetic, and Pytorch.

Citation

If you find this repository useful for your research, please consider starring ⭐ our repo and citing our paper.

@article{huang2024goal,
  title={Goal-Guided Transformer-Enabled Reinforcement Learning for Efficient Autonomous Navigation},
  author={Huang, Wenhui and Zhou, Yanxin and He, Xiangkun and Lv, Chen},
  journal={IEEE Transactions on Intelligent Transportation Systems},
  volume={25},
  number={2},
  pages={1832--1845},
  year={2024},
  publisher={IEEE}
}

Preview Simulation

Click the gif to zoom in πŸ”Ž

Video: Sim-to-Real Experiment ↙️

πŸ‘‰ GTRL Sim-to-Real Navigation Experiment Video πŸ‘ˆ

Basic Dependency Installation

1️⃣ ROS Noetic

2️⃣ Gazebo

3️⃣ Pytorch

User Guidance

Create a new Virtual environment (conda is suggested).

Specify your own name for the virtual environment, e.g., gtrl:

conda create -n gtrl python=3.7

Activate virtual environment.

conda activate gtrl

Install Dependencies.

pip install numpy tqdm natsort cpprb matplotlib einops squaternion opencv-python rospkg rosnumpy yaml
sudo apt install python3-catkin-tools python3-osrf-pycommon
sudo apt-get install ros-noetic-cv-bridge

Optional step for visualizing real-time plotting (reward curve) with Spyder.

conda install spyder==5.2.2

Clone the repository.

cd to your workspace and clone the repo.

git clone https://github.com/OscarHuangWind/DRL-Transformer-SimtoReal-Navigation.git

Compile the workspace.

cd ~/$your workspace/DRL-Transformer-SimtoReal-Navigation/catkin_ws
catkin_make -DPYTHON_EXECUTABLE=/usr/bin/python3

Set up the environment variables.

export GAZEBO_RESOURCE_PATH=~/$your workspace/DRL-Transformer-SimtoReal-Navigation/catkin_ws/src/gtrl/launch
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/ros/noetic/lib

Alternatively, you can select to write these variables to the ~/.bashrc file so that it can be automatically set when opening terminal.

Source the workspace.

source devel/setup.bash

Important!

Copy all the files under models folder to your default gazebo models folder.

cp -a ~/$your workspace/DRL-Transformer-SimtoReal-Navigation/catkin_ws/src/gtrl/models/. ~/.gazebo/models

Revise your system path in main.py and env_lab.py (gtrl/scripts/Environments/env_lab.py) file.

main.py

import sys
sys.path.append('/home/$your workspace/DRL-Transformer-SimtoReal-Navigation/catkin_ws/src/gtrl/scripts')

env_lab.py (line 129)

fullpath = os.path.join('/home/$your workspace/DRL-Transformer-SimtoReal-Navigation/catkin_ws/src/drl_navigation/launch', launchfile)

Time to train and get your GTRL model!!!

cd ~/$your workspace/DRL-Transformer-SimtoReal-Navigation/catkin_ws/src/gtrl/scripts/SAC

Run it in the terminal:

python main.py

(Optional) Alternatively, if you have already installed spyder, just click the run file button in spyder.

To kill the program, it is suggested to use following commands.

killall -9 rosout roslaunch rosmaster gzserver nodelet robot_state_publisher gzclient python python3

Alternatively, you can add alias of these commands to the ~/.bashrc file:

alias k9='killall -9 rosout roslaunch rosmaster gzserver nodelet robot_state_publisher gzclient python python3'

And type the alias in the terminal to kill all the process:

k9

Framework

Goal-guided Transformer (GoT)

Noise-augmented RGB images from fisheye camera

AGV and lab environment model in simulation and real world.

Sim-to-Real navigaiton experiment in office environment.

Releases

No releases published

Packages

No packages published