Language: English | 简体中文

Introduction

MetDetPy is a Python-based meteor detector project that detects meteors from videos and images. Its video detection is inspired by uzanka/MeteorDetector. MetDetPy is powerful and reliable, with the following features:

• Easy-to-use and Configurable: MetDetPy is designed with easy-to-use default settings, so it can be used directly without detailed configuration under most circumstances. Settings are also supported to be changed to get better detection results.

• Applicable for Various Devices and Exposure Time: MetDetPy can detect meteors from video files captured by various types of devices. With adaptive algorithms and deep learning models, MetDetPy works fine for videos and images from full-frame cameras and monitor cameras.

• Low CPU Usage: MetDetPy works with relatively low CPU usage and supports multi-camera real-time detection on mainstream computers or barebones.

• Optional Deep Learning Model: MetDetPy has optional deep learning support. It can be selectively used in the main detection or recheck phase to improve detection results without significantly increasing performance overhead. The model can also be used for meteor detection in images.

• Effective Filter: Meteors will be rechecked according to visual appearance and motion properties to exclude false positive samples. Every prediction is given a confidence score ranging [0,1] which indicates the possibility of being considered a meteor.

• Support Tools: An evaluation tool and a video clip toolkit are also provided to support further video clipping, image stacking, or result evaluation.

Release Version

You can get the latest release version of MetDetPy here. The release version is already packed and can run on common platforms (including Windows, macOS, and Linux) respectively. Also, you can build it yourself with nuitka or pyinstaller (see Package python codes to executables).

Besides, MetDetPy has worked as the backend of the Meteor Master since version 1.2.0. Meteor Master (AI) is a meteor detection software developed by 奔跑的龟斯, which has a well-established GUI, live streaming video support, convenient export function, automatic running, etc. You can get more information at Meteor Master Official Site, or get its latest version from the Microsoft Store / App Store. Its earlier version can get from Baidu NetDisk (Access Code: jz01)

Requirements

Enviroments

• 64bit OS
• Python>=3.7 (3.9+ is recommended)

Requirements

• numpy>=1.15.0
• opencv_python>=4.9.0
• tqdm>=4.0.0
• easydict>=1.0
• multiprocess>=0.70.0
• onnxruntime>=1.16.0

You can install these requirements using:

pip install -r requirements.txt

GPU Support

The above packages enable MetDetPy to run properly, but the deep learning models are only supported on CPU devices for Windows and Linux users. If you wish to utilize your GPU, you can additionally install or replace the onnxruntime-related libraries as follows:

• Recommended: it is recommended to additionally install onnxruntime-directml. This library utilizes DirectX for model inference acceleration and is suitable for most GPUs (Nvidia, AMD, Intel, etc.).

• Nvidia GPU Users (Advanced): If you are using Nvidia GPUs and have CUDA installed, you can install the matched version of onnxruntime-gpu instead of onnxruntime. This enables CUDA acceleration, which brings higher performance.

⚠️ Notice

• For macOS users, since CoreML model inference acceleration is integrated into the latest version of onnxruntime, there is nothing to do to enable GPU support.

• In the current release version, it is onnxruntime_directml for Windows packages. Default CUDA support will be added when it is ready.

Usage

Run Video Meteor Detector

MetDetPy is the launcher of the video meteor detector, its usage is as follows:

python MetDetPy.py target [--cfg CFG] [--mask MASK] [--start-time START_TIME] [--end-time END_TIME] 
               [--exp-time EXP_TIME] [--mode {backend,frontend}] [--debug]
               [--resize RESIZE] [--adaptive-thre ADAPTIVE_THRE] [--bi-thre BI_THRE | --sensitivity SENSITIVITY]
               [--recheck RECHECK] [--save-rechecked-img SAVE_RECHECKED_IMG]
               [--provider {cpu,default,coreml,dml,cuda}][--live-mode {on,off}][--save SAVE]

Main Arguments

• target: meteor video filename. Support common video encoding like H264, HEVC, etc.

• --cfg: path to the configuration file. Use ./config/m3det_normal.json under the config folder by default.

• --mask: mask image. To create a mask image, draw mask regions on a blank image using any color (except white). Support JPEG and PNG format.

• --start-time: the time at which the detection starts (an int in ms or a string format in "HH:MM:SS"). The default value is the start of the video (i.e., 0).

• --end-time: the time until which the detecting ends (an int in ms or a string format in "HH:MM:SS"). The default value is the end of the video.

• --mode: the running mode. Its argument should be selected from {backend, frontend}. In frontend mode, there will be a progress bar indicating related information. In backend mode, the progress information is flushed immediately to suit pipeline workflow. The default is "frontend".

• --debug: indicates whether to print debug information.

• --visu: showing a debug window displaying videos and detected meteors.

• --live-mode: when running in live mode, the detection speed will closely match the actual video time. This option balance cpu cost. Should be selected from {on, off}.

• --provider: specifies the preferred provider to be used for models. The available providers may vary depending on the platform. If the specified provider is not available, the "default" option will be used.

• --save: save detection results (a list) to a json file.

Extra Arguments

The following arguments have default values in config files. If they are configured in command line arguments, the default value will be overrided. Their detailed explanation can be seen in configuration documents.

• --resize: the frame image size used during the detection. This can be set by single int (like 960, for the long side), list (like [960,540]) or string (like 960x540 or 1920x1080).

• --exp-time: the exposure time of each frame in the video. Set with a float number or select from {auto, real-time, slow}. For most cases, option "auto" works well.

• --adaptive-thre: indicates whether apply adaptive binary threshold in the detector. Select from {on, off}.

• --bi-thre: the binary threshold used in the detector. When the adaptive binary threshold is applied, this option is invalidated. Do not set --sensitivity with this at the same time.

• --sensitivity: the sensitivity of the detector. Select from {low, normal, high}. When adaptive binary threshold is applied, higher sensitivity will estimate a higher threshold. Do not set --bi-thre with this at the same time.

• --recheck: indicates whether apply recheck mechanism. Select from {on, off}.

• --save-rechecked-img: the path where rechecked images are saved to.

Example

python MetDetPy.py "./test/20220413Red.mp4" --mask "./test/mask-east.jpg"

Customize Configuration

MetDetPy reads arguments from configuration files. For most circumstances, preset configuration files work well, but there are also times when better detection results can be achieved by adjusting detection arguments. This document explains the meanings of arguments so that they can adjusted according to the requirement. See configuration documents for more information.

Run Image Meteor Detector

To launch the image meteor detector, use run_model.py as follows:

python run_model.py target 

(WIP)

Usage of Other Tools

ClipToolkit

ClipToolkit can be used to create several video clips or stacked images at once. Its usage is as follows:

python ClipToolkit.py target json [--mode {image,video}] [--suffix SUFFIX] [--save-path SAVE_PATH] [--resize RESIZE] [--jpg-quality JPG_QUALITY] [--png-compressing PNG_COMPRESSING] 

Arguments:

• target: the target video.

• json: a JSON-format string or the path to a JSON file where start time, end-time and filename (optional) are listed.

  Specifically, this JSON should be an array of elements, where every element should include at least a "time" key. The value of the "time" key should be an array of two "hh:mm:ss.ms" format strings, which indicates the start time and the end time of the clip. "filename" is an optional key, in whose value you can specify the filename and suffix (i.e what the video clip should be converted to and named.) "filename" is more prior than --mode and --suffix options, but if not specified, this clip will be automatically converted and named according to the command options.

  We provide clip_test.json as a use case and test JSON.

• --mode: convert clip(s) to images or videos. Should be selected from {image, video}. This option will be covered by a specific filename in json.

• --suffix: the suffix of the output. By default, it is "jpg" for image mode and "avi" for video mode. This option will be covered by a specific filename in JSON.

• --save-path: the path where image(s)/video(s) are placed. When only one clip is provided in JSON, you can include the filename in --save-path to simplify your JSON.

• --resize: resize image/video to the given resolution. It should be a string where two numbers are joined by x,like 960x540 or 1920x1080.

• --png-compressing: the compressing rate of the generated png image. It should be int ranged $Z \in [0,9]$; By default, it is 3.

• --jpg-quality: the quality of generated jpg image. It should be int ranged $Z \in [0,100]$; By default, it is 95.

For example:

python ClipToolkit.py "./test/20220413Red.mp4" "./test/clip_test.json" --mode image --suffix jpg --jpg-quality 60 --resize 960x540

Notice: if using a JSON-format string instead of the path to a JSON file, you should be really careful about the escape of double quotes in command lines.

Evaulate

To evaluate how MetDetPy performs on your video, you can simply run evaluate.py :

python evaluate.py target [--cfg CFG] [--load LOAD] [--save SAVE] [--metrics] [--debug] video_json

Arguments

• video_json: a JSON file that places the name of the video, the mask, and meteor annotations. It should be formatted like this:

{
    "video": "path/to/the/video.mp4",
    "mask": "path/to/the/mask.jpg",
    "meteors": [{
        "start_time": "HH:MM:SS.XX0000",
        "end_time": "HH:MM:SS.XX0000",
        "pt1": [
            260,
            225
        ],
        "pt2": [
            154,
            242
        ]
    }]
}

If there is no corresponding mask, simply use "". If there is no meteor annotation, the "meteors" can be ignored too.

• --cfg: configuration file. Use config.json under the same path default.

• --load: the filename of the detection result that is saved by evaluate.py. If it is applied, evaluate.py will directly load the result file instead of running detection through the video.

• --save: the filename of the detection result that is going to save.

• --metrics: calculate precision and recall of the detection. To apply this, "meteors" has to be provided in video_json.

• --debug: when launching evaluate.py with this, there will be a debug window showing videos and detected meteors.

Example

python evaluate.py "test/20220413_annotation.json"

Package python codes to executables

We provide make_package.py to freeze MetDetPy programs into stand-alone executables. This tool supports to use pyinstaller or nuitka to package/compile MetDetPy (and related tools).

When using it, make sure that either pyinstaller or nuitka is installed. Besides, when using nuitka as the packaging tool, make sure that at least one C/C++ compiler is available on your computer.

Its usage is as follows:

python make_package.py [--tool {nuitka,pyinstaller}] [--mingw64]
     [--apply-upx] [--apply-zip] [--version VERSION]

• --tool: your compile/package tool. It should be selected from {nuitka,pyinstaller}. nuitka is the default option.

• --mingw64: use the mingw64 compiler. Only worked when using nuitka and your OS is windows.

• --apply-upx: apply UPX to squeeze the size of the executable program. Only worked when using nuitka.

• --apply-zip: generate zip package when compiling/packaging is finished.

• --version: MetDetPy version tag. Used for naming zip package.

The target executable file and its zip package version (if applied) will be generated in dist directory.

Notice:

1. It is suggested to use Python>=3.9, pyinstaller>=5.0, and nuitka>=1.3.0 to avoid compatibility issues. You can prepare either tool to package the program.
2. Due to the feature of Python, neither tools above can generate cross-platform executable files.
3. If matplotlib or scipy is in the environment, they are likely to be packaged into the final directory together. To avoid this, it is suggested to use a clean environment when packaging.

Performance and Efficiency

1. When applying default configuration on 3840x2160 10fps video, MetDetPy detect meteors with a 20-30% time cost of video length on average (tested with an Intel i5-7500). Videos with higher FPS may cost more time.

2. We test MetDetPy with videos captured from various devices (from modified monitoring cameras to digital cameras), and MetDetPy achieves over 80% precision and over 80% recall on average.

3. MetDetPy now is fast and efficient at detecting most meteor videos. However, when facing complicated weather or other affect factors, its precision and recall can be to be improved. If you find that MetDetPy performs not well enough on your video, it is welcome to contact us or submit issues (if possible and applicable, provide full or clipped video).

License

This project is licensed under the Mozilla Public License 2.0 (MPL-2.0). This means you are free to use, modify, and distribute this software with the following conditions:

1. Source Code Availability: Any modifications you make to the source code must also be made available under the MPL-2.0 license. This ensures that the community can benefit from improvements and changes.
2. File-Level Copyleft: You can combine this software with other code under different licenses, but any modifications to the MPL-2.0 licensed files must remain under the same license.
3. No Warranty: The software is provided "as-is" without any warranty of any kind, either express or implied. Use it at your own risk.

For more detailed information, please refer to the MPL-2.0 license text.

Appendix

Special Thanks

uzanka [Github]

奔跑的龟斯 [Personal Website] [Weibo] [Bilibili]

纸片儿 [Github]

DustYe夜尘 [Bilibili]

RoyalK [Weibo] [Bilibili]

MG_Raiden扬 [Weibo] [Bilibili]

星北之羽 [Bilibili]

LittleQ

韩雅南

来自偶然

杨雳鹏

兔爷 [Weibo][Bilibili]

Jeff戴建峰 [Weibo] [Bilibili]

贾昊

Update Log / Todo List

See update log.

Statistics