How to save Video (from webcam) and Audio (from directsound device) in avi file ?

[yoannwyffels]

What would you like to submit? (put an 'x' inside the bracket that applies)

• question
• bug report
• feature request

Issue description

Hello,

I'm trying to accomplish something easy as first sight: recording video and audio (from webcam) in an avi file. It appears more complex that it seems :)

So far, I handle video frames nicely (I think) with this frame handler:

void videoDevice_NewFrame(object sender, Accord.Video.NewFrameEventArgs eventArgs)
        {

            System.Drawing.Image imgforms = (Bitmap)eventArgs.Frame.Clone();
            BitmapImage bi = new BitmapImage();
            bi.BeginInit();

            MemoryStream ms = new MemoryStream();
            imgforms.Save(ms, ImageFormat.Bmp);
            ms.Seek(0, SeekOrigin.Begin);

            bi.StreamSource = ms;
            bi.EndInit();

            //Using the freeze function to avoid cross thread operations 
            bi.Freeze();

            //Calling the UI thread using the Dispatcher to update the 'Image' WPF control         
            Dispatcher.BeginInvoke(new ThreadStart(delegate
            {
                pbox.Source = bi; /*pbox is the name of the 'Image' WPF control*/
            }));

            if (_recording)
            {
                long currentTick = DateTime.Now.Ticks;
                StartTick = StartTick ?? currentTick;
                var frameOffset = new TimeSpan(currentTick - StartTick.Value);

                double elapsedTimeInSeconds = stopwatch.ElapsedTicks / (double)Stopwatch.Frequency;
                double timeBetweenFramesInSeconds = 1.0 / 25;
                if (elapsedTimeInSeconds >= timeBetweenFramesInSeconds)
                {
                    stopwatch.Restart();
                    try
                    {
                        _writer.WriteVideoFrame(eventArgs.Frame, frameOffset);
                    }catch(Exception ex)
                    {

                    }
                    
                    //Here come WriteAudioFrame(Signal signal) ?
                }
                
            }

        }

and Audio data is stored in a memory stream / WaveEncoder in the audio new frame event handler:

audio_stream = new MemoryStream();           
audio_encoder = new WaveEncoder(audio_stream);

void audioDevice_NewFrame(object sender, Accord.Audio.NewFrameEventArgs eventArgs)
        {

            audio_encoder.Encode(eventArgs.Signal);
            
        }

So now, I don't understand how to transform audio memory stream (or WaveEncoder ?) into Signal object needed to new method writted by @cesarsouza :
_writer.WriteAudioFrame(Signal signal)
and how to synchronise video and audio. I suspect we need to get same amount of audio and video frames according to the samplerate.

Do you have any exemple how to do that ?

Many thanks

[hbtech-ai]

Hi,yoannwyffels
Have you solved the problem? I have the same problem.

[yoannwyffels]

Hi hbtech-ai, Yes and No...I've finally record video and audio separatly, and then merge them at the end of recording with ffmpeg.exe (and eventually do some more compression with it).

[hbtech-ai]

Hi,yoannwyffels
Thanks for your answer,you means video and audio should separatly recorded,then merge audio and video using ffmpeg.exe,I will test it soon.

[yoannwyffels]

Correct.

[hbtech-ai]

Hi,yoannwyffels,
I test ffmpeg merge video and audio,but video length is below the audio length,so merge result is video stop in a moment,audio is play.how you to reslove this problem.

[yoannwyffels]

You have to write video frames with frameOffset informations (how many times the frame is). You just have to use a timespan to do this in your c# code.

private long? StartTick;

void audioDevice_NewFrame(object sender, Accord.Audio.NewFrameEventArgs eventArgs)
        {

            if (_recording)
            {
                audio_encoder.Encode(eventArgs.Signal);
            }

        }


void videoDevice_NewFrame(object sender, NewFrameEventArgs eventArgs)
        {

            

            if (_recording)
            {

                long currentTick = DateTime.Now.Ticks;
                StartTick = StartTick ?? currentTick;
                var frameOffset = new TimeSpan(currentTick - StartTick.Value);
                try
                {
                    _writer.WriteVideoFrame(eventArgs.Frame, frameOffset);
                }catch(Exception ex)
                {
                    App.logit(ex.Message);
                }

            }
}

For what I remember, some compression codecs aren't working well.
Here is how I configured the videoFileWrite (I'm using Video and Audio "Default" Codec.

public void startRecording(string previewVideo_www)
        {

            previewVideo_whileRec = previewVideo_www;

            if (_recording== false)
            {
                StartTick = null;
                _writer = new VideoFileWriter()
                {
                    Width = video_vc.FrameSize.Width,
                    Height = video_vc.FrameSize.Height,
                    FrameRate = video_vc.AverageFrameRate,
                    VideoCodec = VideoCodec.Default,
                    AudioCodec = AudioCodec.Default,
                    AudioBitRate = 44100,
                    BitRate = 3000 * 1000,
                    AudioLayout = AudioLayout.Mono,
                    

                };
                _writer.Open(App.DataPath + "temp.avi");

                audio_file_stream = new FileStream(App.DataPath + "temp.wav", FileMode.Create);
                audio_encoder = new WaveEncoder(audio_file_stream);

                final_videoFileName = "MyVideo-"+DateTime.Now.ToString("yyyyMMdd-HHmmss");
                _recording = true;
            }
                

            
        }

and here is the ffmpeg command line I use to merge (using copy method, no compression):

private void mergeAudioAndVideo()
        {

            App.logit("Merging Audio & Video");
            File.Delete(App.DataPath + "merge.avi");
           
            Process pss = new Process();
            ProcessStartInfo si = new ProcessStartInfo();
            si.FileName = App.ExePath + "ffmpeg.exe";
            si.UseShellExecute = false;
            si.CreateNoWindow = true;
            si.Arguments = "-i \""+App.DataPath+"temp.avi\" -i \""+App.DataPath+"temp.wav\" -c:v copy -c:a copy -map 0:v:0 -map 1:a:0 \""+App.DataPath+ final_videoFileName+".avi\"";

            pss.StartInfo = si;
            pss.Start();
            pss.WaitForExit();

            if (File.Exists(App.DataPath + final_videoFileName+".avi"))
            {
                File.Delete(App.DataPath + "temp.wav");
                File.Delete(App.DataPath + "temp.avi");
                App.logit("record succeed !");
                File.Move(App.DataPath + final_videoFileName + ".avi", GetFolderSave() + "\\" + final_videoFileName + ".avi");
                lastCapturedMedia = GetFolderSave() + "\\" + final_videoFileName + ".avi";
            }

        }

I hope it will help you well :)

[hbtech-ai]

Thanks for your help.I will test it soon.

[bbrown-adig]

Did anyone get this to work?

[elboudna]

@yoannwyffels Hello Yoan Would you please send me the full code please at : mehdi_boudnaoui@hotmail.fr, i'm struggling with the audio part.
Thanks.

[MohammadNsr]

hey
can some one post the complete project. it will be helpful to see a complete sample of using it.

[ganyanchuan1989]

@yoannwyffels Would you please send me the full code please at : ganxunzou@163.com .
Thanks.