this is sample code of warping background image in vuforia for stereo rectify. vuforia example provide two ways of accessing the camera image. While the first image class example leads low frame raw, we use the second approach, shader program warps the video background directly. The opencv precompute maps, lookup coordinate for warping is passed into the shader program as another texture. The result is cool. I achieve up to 60fps frame rate in my environment.
before:
rectified:
a more detail code explaination could be found on my blog
- /save_param/ keeps the calibration output files from opencv stereo calibration example like this, we only use the precompute maps calib_para.yml, make sure it's in CV_16S2 formate. change these files to your calibration reslut.
- I have two cameras with resolution 960*1080. Change this to yours in some of the codes and the vuforia setup xml.
- /opencv_dll/ is a sample opencv dll project, producing dlls for unity plugin. I use opencv to read the yml files and change the map from CV_16S2 to CV_32FC1. opencv 2.4.11 for win32 is used. recompile the dlls and copy them to the /assects/plugins/ folder if you have different environment(but vuforia could only works with Unity 32bit).