Benchmarking Fish Dataset and Evaluation Metric in Keypoint Detection -Towards Precise Fish Morphological Assessment in Aquaculture Breeding

This repository contains the code and dataset for the paper "Benchmarking Fish Dataset and Evaluation Metric in Keypoint Detection -Towards Precise Fish Morphological Assessment in Aquaculture Breeding," accepted at IJCAI2024 in the AI and Social Good track.

Weizhen Liu¹, Jiayu Tan¹, Guangyu Lan¹, Ao Li¹, Dongye Li⁴, Le Zhao¹, Xiaohui Yuan^{1,3 *}, Nanqing Dong^{2 *}

¹School of Computer Science and Artificial Intelligence, Wuhan University of Technology

²Shanghai Artificial Intelligence Laboratory

³Yazhouwan National Laboratory

⁴Sanya Boruiyuan Technology Co. Ltd

^{{liuweizhen, tjy2023305211,yuanxiaohui}@whut.edu.cn, dongnanqing@pjlab.org.cn}

Overview

• Motivation and design
• Dataset
• Environment Setup
• Citations

Motivation and design

Accurate phenotypic analysis in aquaculture breeding necessitates detailed morphological data, yet current datasets are small-scale, species-limited, and lack comprehensive annotations. To address these shortcomings, we introduce FishPhenoKey, a dataset of 23,331 high-resolution images across six fish species, annotated with 22 phenotype-oriented keypoints. This dataset facilitates precise measurement of complex morphological features critical for advanced aquaculture research.

FishPhenoKey is designed to support diverse research needs with flexibility and scalability. By integrating high-resolution imaging and robust annotation protocols, it provides the necessary diversity and detail for accurate phenotypic analysis. The modular design of FishPhenoKey ensures compatibility with various evaluation metrics and keypoint detection models, enabling continuous updates and expansion to accommodate future advancements in aquaculture and genetic studies.

Dataset

The FishPhenoKey dataset includes high-resolution JPG images of six fish species, each captured from two different viewpoints. Under the guidance of fishery experts, we have precisely annotated four of these species to support the task of measuring subtle morphological phenotypes in fish body parts. The definitions of the 22 keypoints are detailed as follows: 1: tip of the snout, 2: posterior end of the operculum, 3: top end of the head, 4: isthmus, 5: dorsal apex, 6: bottom end of the ventral margin, 7: top end of the caudal peduncle, 8: bottom end of the caudal peduncle, 9: posterior end of the tail fin, 10: posterior end of the caudal vertebrae, 11: anterior end of the eye, 12: posterior end of the eye, 13: anterior end of the pectoral fin, 14: posterior end of the pectoral fin, 15: anterior end of the pelvic fin, 16: posterior end of the pelvic fin, 17: anterior end of the anal fin, 18: posterior end of the anal fin, 19: outer margin of the anal fin, 20: anterior end of the dorsal fin, 21: posterior end of the dorsal fin, 22: outer margin of the dorsal fin.

If you need the complete dataset, please download the FishPhenoKey Dataset User Agreement and read the relevant regulations. If you agree to the regulations, please fill in the relevant user information in the user agreement, [authorization date], and [electronic signature] at the end of the agreement. Send the PDF format of the user agreement to the email liuweizhen@whut.edu.cn. After review, we will send the download link for the complete dataset via email.

Environment Setup

• Python 3.6/3.7/3.8
• Pytorch 1.10 or above
• pycocotools (Linux: pip install pycocotools; Windows: pip install pycocotools-windows (no need to install VS separately))
• Ubuntu or CentOS (Windows not recommended)
• Preferably use GPU for training
• For detailed environment setup, refer to requirements.txt

Citations

Please cite our paper in your publications if our methods and dataset are helpful to your research. The BibTeX is as follows:

@article{liu2024benchmarking,
  title={Benchmarking Fish Dataset and Evaluation Metric in Keypoint Detection-Towards Precise Fish Morphological Assessment in Aquaculture Breeding},
  author={Liu, Weizhen and Tan, Jiayu and Lan, Guangyu and Li, Ao and Li, Dongye and Zhao, Le and Yuan, Xiaohui and Dong, Nanqing},
  journal={arXiv preprint arXiv:2405.12476},
  year={2024}
}