Nan value in the nearest points when using distance()

[tongtybj]

While doing distance check for two boxes, I found nan values in the nearest points after using distance().
One of the sample is as follows, and two boxes do not collide.

   DistanceResult<double> result;
   result.clear();
   DistanceRequest<double> request(true);

   std::shared_ptr< CollisionGeometry<double> > cgeomBox1(new Box<double>(2.750000, 6.000000, 0.050000));
   std::shared_ptr< CollisionGeometry<double> > cgeomBox2(new Box<double>(0.424000, 0.150000, 0.168600));
   CollisionObject<double> objBox1(cgeomBox1, Eigen::Quaterniond(1,0,0,0).matrix(), Eigen::Vector3d(1.625000, 0.000000, 0.500000));
   CollisionObject<double> objBox2(cgeomBox2, Eigen::Quaterniond(0.672811, 0.340674, 0.155066, 0.638138).matrix(), Eigen::Vector3d(0.192074, -0.277870, 0.273546));

   fcl::distance(&objBox1, &objBox2, request, result);
   std::cout << "distance = " << result.min_distance << std::endl;
   std::cout << " point on obj1: x = " << result.nearest_points[0](0) << " y = " << result.nearest_points[0](1) << " z = " << result.nearest_points[0](2) << std::endl;
   std::cout << " point on obj2: x = " << result.nearest_points[1](0) << " y = " << result.nearest_points[1](1) << " z = " << result.nearest_points[1](2) << std::endl;

Can someone help me to figure out the reason, or give me some hint about the algorithm of distance calculation between two boxes?

[SeanCurtis-TRI]

Thanks for the detailed reproduction info. I'll look into this and get back to you.

[tongtybj]

@SeanCurtis-TRI
Thank you for your response.

What I have found is that the normal n is zero, whiling processing the function extractClosestPoints, and this is caused since vector AB and AC are parallel. Right now, I am studying about the algorithm to figure out why triangle ABC is corrupt.