Skip to content

Latest commit

 

History

History

benchmark

README.md

This application is used to check everything is ok and running as fast as expected, check the C++ API for more information. It's open source and doesn't require registration or license key.

Dependencies

The SDK is developed in C++11 and you'll need glibc 2.27+ on Linux and Microsoft Visual C++ 2015 Redistributable(x64) - 14.0.24123 (any later version is ok) on Windows. You most likely already have these dependencies on you machine as almost every program require it.

If you're planning to use OpenVINO, then you'll need Intel C++ Compiler Redistributable (choose newest). Please note that OpenVINO is packaged in the SDK as plugin and loaded (dlopen) at runtime. The engine will fail to load the plugin if Intel C++ Compiler Redistributable is missing on your machine but the program will work as expected with Tensorflow as fallback. We highly recommend using OpenVINO to speedup the inference time and reduce memory usage.

Debugging missing dependencies

To check if all dependencies are present:

GPGPU acceleration

  • On x86-64, GPGPU acceleration is disabled by default. Check here for more information on how to enable it.

Peformance numbers

These performance numbers are obtained using version 0.0.1. You can use any later version.

We ran the benchmark application for #20 times loops United States - California Driving License (2017).jpg file.

The first number (5727 millis, 3.49fps) means it takes 286 milliseconds to fully process the Californian driver license on RTX3060. 286 = 1000/3.49.

Inter Parallel processing enabled Inter Parallel processing disabled
AMD Ryzen 7 3700X 8-Core
RTX 3060
Ubuntu 20		 5727 millis
3.49 fps		 11332 millis
1.76 fps
Intel(R) Xeon(R) E3-1230 v6 @ 3.50GHz
GTX 1070
Ubuntu 18		 8142 millis
2.45 fps		 14480 millis
1.38 fps
Intel(R) i7-4790K @4.40GHz
No GPU
Windows 8 Pro		 24587 millis
0.81 fps		 24776 millis
0.80 fps

Some important notes:

  • All tests are done with OpenVINO activation mode set to "auto". You'll have very poor performance numbers if you disable OpenVINO without having a GPU. Another reason th use OpenVINO instead of Tensorflow is that the former consumes far less memory.
  • Set OpenVINO activation mode to "on" instead of "auto" if you have a GPU but don't want to use it. "on" will force all inference to be done on OpenVINO device (default = "CPU").
  • Support for CUDA is checked at runtime, check the logs to make sure evrything is ok.
  • Inter parallel processing mode is faster than sequential mode only when you have a GPU or NPU. More at https://www.doubango.org/SDKs/kyc-documents-verif/docs/Parallel_processing.html#inter-processing

Pre-built binaries

If you don't want to build this sample by yourself, then use the pre-built versions:

Building

You'll need CMake to build this sample.

  • Create build folder and move into it: mkdir build && cd build

To generate the build files:

  • Windows (Visual Studio files): cmake .. -DCMAKE_BUILD_TYPE=Release
  • Linux (Makefile): cmake .. -G"Unix Makefiles" -DCMAKE_BUILD_TYPE=Release

To build the project:

  • Windows: Open the VS solution and build the projet
  • Linux: Run make to build the project

Testing

After building the application you can test it on your local machine.

The test image looks like this:

Test image

Usage

Benchmark is a command line application with the following usage:

benchmark \
      --image <path-to-image-to-process> \
      --assets <path-to-assets-folder> \
      [--loops <number-of-loops>] \
      [--vino_activation <openvino-activation-mode:auto/on/off>] \
      [--parallel <whether-to-enable-inter-parallel-mode:true/false>] \
      [--gpu_ctrl_mem <whether-to-enable-gpu-memory-ctrl:true/false>] \
      [--tokenfile <path-to-license-token-file>] \
      [--tokendata <base64-license-token-data>]

Options surrounded with [] are optional.

Examples

  • On Linux x86_64, you may use the next command:
LD_LIBRARY_PATH=../../../binaries/linux/x86_64:$LD_LIBRARY_PATH ./benchmark \
    --image "../../../assets/images/United States - California Driving License (2017).jpg" \
    --assets ../../../assets \
    --loops 20 \
    --vino_activation "auto" \
    --gpu_ctrl_mem false \
    --parallel true

Very important: you'll need to download Tensorflow libraries as explained here.

you can also use binaries/linux/x86_64/benchmark.sh to make your life easier.

  • On Windows x86_64, you may use the next command:
benchmark.exe ^
    --image "../../../assets/images/United States - California Driving License (2017).jpg" ^
    --assets ../../../assets ^
    --loops 20 ^
    --vino_activation "auto" ^
    --gpu_ctrl_mem false ^
    --parallel true

you can also use binaries/windows/x86_64/benchmark.bat to make your life easier.