10.GLSurfaceView+MediaPlayer播放视频

视频播放

平时的视频播放都是使用mediaplayer+textureview或者mediaplayer+surfaceview，但是如果我们要对视频进行一些OpenGL的操作，打个比方说我要在视频播放的时候添加一个滤镜，这个时候就需要用到SurfaceTexture了。

大体步骤如下:

GLSurfaceView -> setRender -> onSurfaceCreated回调方法中构造一个SurfaceTexture对象，然后设置到Camera预览或者MediaPlayer中 -> SurfaceTexture中的回调方法onFrameAvailable来得知一帧的数据准备完成 -> requestRender通知Render来绘制数据 -> 在Render的回调方法onDrawFrame中调用SurfaceTexture的updateTexImage方法来获取一帧数据，然后开始使用GL进行绘制预览。

具体步骤:

在GLSurfaceView.Render中创建一个纹理，再使用该纹理创建一个SurfaceTexture。
使用该SurfaceTexture创建一个Surface传给相机，相机预览数据就会输出到这个纹理上了。
使用GLSurfaceView.Render将该纹理渲染到GLSurfaceView的窗口上。
使用SurfaceTexture的setOnFrameAvailableListener方法给SurfaceTexture添加一个数据帧可用的监听器，在监听器中调用GLSurfaceView的requestRender方法渲染该帧数据，这样相机每次输出一帧数据就可以渲染一次，在GLSurfaceView窗口中就可以看到相机的预览数据了。

顶点着色器

#version 300 es
in vec4 vPosition;
in vec2 vCoordPosition;
out vec2 aCoordPosition;

void main() {
    gl_Position = vPosition;
    aCoordPosition = vCoordPosition;
}

片段着色器

这里需要注意一下，就是做相机预览和视频播放的话，纹理的类型需要使用samplerExternalOES，而不是之前渲染图片的sampler2D。这是因为相机和视频的数据是YUV的，而OpenGL ES是RGB的，samplerExternalOES内部会进行处理。#extension用于启用和设置扩展的行为。格式为#extension all : behavior。behavior的可选值有: require、enable、warn、disable。
```
#version 300 es
#extension GL_OES_EGL_image_external_essl3 : require
precision mediump float;
in vec2 aCoordPosition;
uniform samplerExternalOES uSamplerTexture;
out vec4 vFragColor;
void main() {
    vFragColor = texture(uSamplerTexture, aCoordPosition);
}
```

GLSurfaceView+MediaPlayer播放视频

public class VideoPlayerActivity extends Activity {
    private GLSurfaceView mGLSurfaceView;
    private Surface mSurface;
    private MediaPlayer mMediaPlayer;

    @Override
    protected void onCreate(@Nullable Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_video_player);
        mGLSurfaceView = findViewById(R.id.mGLSurfaceView);
        mGLSurfaceView.setEGLContextClientVersion(3);
        VideoPlayerRender videoPlayerRender = new VideoPlayerRender(mGLSurfaceView);
        videoPlayerRender.setIVideoTextureRenderListener(new VideoPlayerRender.IVideoTextureRenderListener() {
            @Override
            public void onCreate(SurfaceTexture surfaceTexture) {
                mSurface = new Surface(surfaceTexture);
                startPlay();
            }
        });
        mGLSurfaceView.setRenderer(videoPlayerRender);
    }

    @Override
    protected void onResume() {
        super.onResume();
        if (mSurface != null && mSurface.isValid()) {
            startPlay();
        }
        if (mGLSurfaceView != null) {
            mGLSurfaceView.onResume();
        }
    }

    private void startPlay() {
        if (mMediaPlayer != null && mSurface != null && mSurface.isValid()) {
            mMediaPlayer.setSurface(mSurface);
            mMediaPlayer.start();
            return;
        }
        mMediaPlayer = new MediaPlayer();
        try {
            mMediaPlayer = MediaPlayer.create(this, R.raw.beauty);
            mMediaPlayer.setOnPreparedListener(new MediaPlayer.OnPreparedListener() {
                @Override
                public void onPrepared(MediaPlayer mp) {
                    if (mp != null) {
                        mp.start();
                    }
                }
            });
            mMediaPlayer.setOnErrorListener(new MediaPlayer.OnErrorListener() {
                @Override
                public boolean onError(MediaPlayer mp, int what, int extra) {
                    return false;
                }
            });
            // 将surface设置给mediaplayer
            mMediaPlayer.setSurface(mSurface);
            mSurface.release();
            mMediaPlayer.setScreenOnWhilePlaying(true);
            mMediaPlayer.setLooping(true);
            mMediaPlayer.prepareAsync();
        } catch (Exception e) {
            e.printStackTrace();
        }
    }

    @Override
    protected void onPause() {
        super.onPause();
        if (mGLSurfaceView != null) {
            mGLSurfaceView.onPause();
        }
        if (mMediaPlayer != null && mMediaPlayer.isPlaying()) {
            mMediaPlayer.pause();
        }
    }

    @Override
    protected void onDestroy() {
        super.onDestroy();
        if (mMediaPlayer != null) {
            mMediaPlayer.reset();
            mMediaPlayer.release();
            mMediaPlayer = null;
        }
    }
}

视频是能播起来了，但是变形了

修复变形

我们要根据视频的宽高和Render中Surface的宽高来计算缩放的比例，让高度或者宽度进行缩放，这样就不会变形了。主要修改的地方就是通过换算这个比例然后根据这个比例来改变顶点渲染器中的点的坐标值，从而让其在换算后的坐标内开始画面，这样就不会变形了。

下面是改动后的VideoPlayerRender类:

public class VideoPlayerRender extends BaseGLSurfaceViewRenderer {
    private int mTextureId;
    private SurfaceTexture mSurfaceTexture;
    private GLSurfaceView mGLSurfaceView;
    private boolean mUpdateSurfaceTexture;
    private FloatBuffer mVertextBuffer;
    private FloatBuffer mTextureBuffer;
    private int vertexPosition;
    private int texturePosition;
    private int samplerTexturePosition;
    /**
     * 视频的宽高
     */
    private int mVideoWidth;
    private int mVideoHeight;
    /**
     * 需改更改渲染的大小
     */
    private boolean mNeedUpdateSize;
    /**
     * Surface的宽高
     */
    private int mSurfaceWidth;
    private int mSurfaceHeight;

    /**
     * 坐标占用的向量个数
     */
    private static final int POSITION_COMPONENT_COUNT = 2;
    // 逆时针顺序排列
    private static final float[] POINT_DATA = {
            -1f, -1f,
            1f, -1f,
            -1f, 1f,
            1f, 1f,
    };
    /**
     * 颜色占用的向量个数
     */
    private static final int TEXTURE_COMPONENT_COUNT = 2;
    // 纹理坐标(s, t)，t坐标方向和顶点y坐标反着
    private static final float[] TEXTURE_DATA = {
            0.0f, 1.0f,
            1.0f, 1.0f,
            0.0f, 0.0f,
            1.0f, 0.0f
    };

    public VideoPlayerRender(GLSurfaceView surfaceView) {
        mGLSurfaceView = surfaceView;
        mVertextBuffer = BufferUtil.getFloatBuffer(POINT_DATA);
        mTextureBuffer = BufferUtil.getFloatBuffer(TEXTURE_DATA);
    }

    @Override
    public void onSurfaceCreated(GL10 gl, EGLConfig config) {
        glClearColor(0.0f, 1.0f, 0.0f, 1.0f);
        handleProgram(MyApplication.getInstance(), R.raw.video_vertex_shader, R.raw.video_fragment_shader);
        vertexPosition = glGetAttribLocation("vPosition");
        texturePosition = glGetAttribLocation("vCoordPosition");
        samplerTexturePosition = glGetUniformLocation("uSamplerTexture");
        mTextureId = TextureUtil.createOESTextureId();
        mSurfaceTexture = new SurfaceTexture(mTextureId);
        mSurfaceTexture.setDefaultBufferSize(mGLSurfaceView.getWidth(), mGLSurfaceView.getHeight());
        mSurfaceTexture.setOnFrameAvailableListener(new SurfaceTexture.OnFrameAvailableListener() {
            @Override
            public void onFrameAvailable(SurfaceTexture surfaceTexture) {
                mUpdateSurfaceTexture = true;
                if (mGLSurfaceView != null) {
                    mGLSurfaceView.requestRender();
                }
            }
        });
        if (mTextureRenderListener != null) {
            mTextureRenderListener.onCreate(mSurfaceTexture);
        }
    }

    @Override
    public void onSurfaceChanged(GL10 gl, int width, int height) {
        mSurfaceWidth = width;
        mSurfaceHeight = height;
        adjustVideoSize();
        Log.e("@@@", "onSurfaceChanged width: " + width + "...height.." + height);
        glViewport(0, 0, width, height);
    }

    @Override
    public void onDrawFrame(GL10 gl) {
        glClear(GLES30.GL_DEPTH_BUFFER_BIT | GLES30.GL_COLOR_BUFFER_BIT);
        adjustVideoSize();
        glVertexAttribPointer(vertexPosition, POSITION_COMPONENT_COUNT, GLES30.GL_FLOAT, false, 0, mVertextBuffer);
        glVertexAttribPointer(texturePosition, TEXTURE_COMPONENT_COUNT, GLES30.GL_FLOAT, false, 0, mTextureBuffer);
        synchronized (this) {
            if (mUpdateSurfaceTexture) {
                mSurfaceTexture.updateTexImage();
                mUpdateSurfaceTexture = false;
            }
        }
        GLES30.glEnableVertexAttribArray(vertexPosition);
        GLES30.glEnableVertexAttribArray(texturePosition);
        GLES30.glUniform1i(samplerTexturePosition, 0);
        // 绘制
        GLES30.glDrawArrays(GLES30.GL_TRIANGLE_STRIP, 0, 4);
        GLES30.glFlush();
        GLES30.glDisableVertexAttribArray(vertexPosition);
        GLES30.glDisableVertexAttribArray(texturePosition);
    }

    public void setVideoSize(int width, int height) {
        if (mVideoWidth == width && mVideoHeight == height) {
            return;
        }
        // videoWidth 272
        // videoHeight 480
        mVideoWidth = width;
        mVideoHeight = height;
        mNeedUpdateSize = true;
    }

    private void adjustVideoSize() {
        if (mVideoWidth == 0 || mVideoHeight == 0 || mSurfaceHeight == 0 || mSurfaceWidth == 0) {
            return;
        }
        if (!mNeedUpdateSize) {
            return;
        }
        mNeedUpdateSize = false;
        float widthRation = (float) mSurfaceWidth / mVideoWidth;
        float heightRation = (float) mSurfaceHeight / mVideoHeight;
        float ration = Math.max(widthRation, heightRation);
        // 把视频宽高最小的一个扩大到Surface的大小
        int targetVideoWidth = Math.round(mVideoWidth * ration);
        int targetVideoHeight = Math.round(mVideoHeight * ration);
        // 扩大之后的宽高除以目前surface的宽高，来算错各自要xy的比例，这俩里面有一个肯定是1

        float rationX = (float) targetVideoWidth / mSurfaceWidth;
        float rationY = (float) targetVideoHeight / mSurfaceHeight;

        float[] targetPositionData = new float[]{
                POINT_DATA[0] / rationY, POINT_DATA[1] / rationX,
                POINT_DATA[2] / rationY, POINT_DATA[3] / rationX,
                POINT_DATA[4] / rationY, POINT_DATA[5] / rationX,
                POINT_DATA[6] / rationY, POINT_DATA[7] / rationX,

        };
        // 换算缩放后的顶点坐标。后面在onDraw()方法中会有这个值设置给顶点着色器
        mVertextBuffer.clear();
        mVertextBuffer.put(targetPositionData);
        mVertextBuffer.position(0);
    }

    private IVideoTextureRenderListener mTextureRenderListener;

    public void setIVideoTextureRenderListener(IVideoTextureRenderListener render) {
        mTextureRenderListener = render;
    }

    public interface IVideoTextureRenderListener {
        void onCreate(SurfaceTexture surfaceTexture);
    }
}

下面是具体的效果分别是填充宽和填充高的效果:

邮箱：charon.chui@gmail.com
Good Luck!

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

10.GLSurfaceView+MediaPlayer播放视频.md

10.GLSurfaceView+MediaPlayer播放视频.md

10.GLSurfaceView+MediaPlayer播放视频

视频播放

GLSurfaceView+MediaPlayer播放视频

修复变形

Files

10.GLSurfaceView+MediaPlayer播放视频.md

Latest commit

History

10.GLSurfaceView+MediaPlayer播放视频.md

File metadata and controls

10.GLSurfaceView+MediaPlayer播放视频

视频播放

GLSurfaceView+MediaPlayer播放视频

修复变形