Multi Cam Object Detection

Installation

Please first install ros2 galactic. Then install required ROS2 packages.

sudo apt install ros-galactic-vision-msg
sudo apt install ros-galactic-cv-bridge
pip install -r requirements.txt 
pip install depthai
pip install opencv-python

After that, please build the package with

colcon build --packages-select multi_cam_obj_detection

and source with

source install/setup.bash

to rebuld the package from scratch please run

rm -rf install
rm -rf build
colcon build

Note: remove install/ and build/ will remove all built packages

Usage

Calibration

During Calibration, Please only connect one camera at a time.

1. Please go to src/multi_cam_obj_detection/config/img_capture.yaml to change the path the iamge will save to. If you want to change the resolution, it is where you will change as well. Note: The maximum resolution is set to 1080P, that is the resolution you pick can not go beyond that for now, if you wish to go up to 4k, please change multi_cam_node.py accordingly. Use ABSOLUTE PATH to save iamge.
2. Run

ros2 launch multi_cam_obj_detection calibration_image_capture.launch.py

After that a window will pop up showing the image, please click on that window and follow instruction prompted in terminal to capture image, the default format is bmp.
3. Note that at this time the camera's MXID will be printed to the terminal, please record that as it will be in later use.

Object Detection and BBOX Publishing

Before starting the whole pipeline, please go to src/multi_cam_obj_detection/launch/cam_launch.py to remap the topics with camera's MXID and to the topic of your preference. NOTE: If you are only using one camera, do not add more than one topic as message_filter will block callback until msg recieved from that topic.

1. Please then go to src/multi_cam_obj_detection/config/cam_sub.yaml and add the remapped to topic to topics as a list. Please also change other parameters accordingly. Please also specify the topics where bounding box is published to. NOTE: All paths should be ABSOLUTE PATH
2. Launch cameras with

ros2 launch multi_cam_obj_detection cam.launch.py

Make sure the camera is running, run

rviz2

and click on add, by topics and select the topics you set to see if images present. 3. Launch detection node

ros2 launch multi_cam_obj_detection sub.launch.py

DOCUMENTATION STILL ONGOING