euler_angles not correct for right-handed coordinate system

[jpvanoosten]

If GLM follows the convention of a right-handed coordinate system then the Euler angle functions in euler_angles.hpp are not correct. Using a right-handed coordinate system positive rotations will rotate an axis in a counter-clockwise direction when looking towards the positive axis of rotation. For example, when looking towards the positive X axis, the Y axis will rotate towards the positive Z axis when a positive rotation is applied and when looking towards the positive Y axis, the X axis will rotate towards the negative Z axis when a positive rotation is applied.

If the Eurler function eulerAngleXY is rotX * rotY then the function should be changed to:

    template <typename T>
    GLM_FUNC_QUALIFIER detail::tmat4x4<T, defaultp> eulerAngleXY
    (
        T const & angleX,
        T const & angleY
    )
    {
        T cosX = glm::cos(angleX);
        T sinX = glm::sin(angleX);
        T cosY = glm::cos(angleY);
        T sinY = glm::sin(angleY);

        return detail::tmat4x4<T, defaultp>(
            cosY,   -sinX * -sinY,  cosX * -sinY,   T(0),
            T(0),   cosX,           sinX,           T(0),
            sinY,   -sinX * cosY,   cosX * cosY,    T(0),
            T(0),   T(0),           T(0),           T(1));
    }

And if the Euler function eulerAngleYX is rotY * rotX then this function should be:

    template <typename T>
    GLM_FUNC_QUALIFIER detail::tmat4x4<T, defaultp> eulerAngleYX
    (
        T const & angleY,
        T const & angleX
    )
    {
        T cosX = glm::cos(angleX);
        T sinX = glm::sin(angleX);
        T cosY = glm::cos(angleY);
        T sinY = glm::sin(angleY);

        return detail::tmat4x4<T, defaultp>(
            cosY,          0,      -sinY,    T(0),
            sinY * sinX,  cosX, cosY * sinX, T(0),
            sinY * cosX, -sinX, cosY * cosX, T(0),
            T(0),         T(0),     T(0),    T(1));
    }

SInce the eulerAngleX, eulerAngleY, and eulerAngleX functions appear to be correct, the functionality of these functions can be verified by replacing eulerAngleYX in your code with eulerAngleX() * eulerAngleY(). If the object rotates in the same direction then they are correct. Currently, this is not the case.

[Groovounet]

Your proposed fix resolve the issue and have been committed into GLM 0.9.5 branch for next release.
I have also added a full test suite for the eulerAngle*** functions.

Thanks for contributing,
Christophe