Transcode while recording

[Spectragr]

Hello,
Has anyone tried starting transcoding while the file is still being recorded? it would be convenient in large registrations 2 hours.

[emcodem]

Hi Spectragr, welcome to the Forum and thank you for using FFAStrans!

yes i have such workflows. In a professional environment, such workflows are not exactly easy to support because it is hard to check if the process actually worked or we got a too short output file (finding the reason for the too short output file is a special topic on its own)

You'll need to fulfill some basics for start:
*) the incoming file format/container can be processed as growing (e.g. mxf, mkv or mp4/mov with faststart)
*) the writing application allows other applications to read the file while it is being written (open file shared for reading)
*) the writing application writes actually a growing file (e.g. windows copy does NOT work for such workflow because it allocates the whole file size before writing the file, ending up in a "not growing" file)
*) the writing application does not attemt to rename the written file when it is done
*) the writing application does not pause writing data to the growing file for more than a second or so
*) the storage, protocol and drivers support such operations (you need to trial and error)

If you fulfill all these requirements, congrats, you can read on
*) For Watchfolder based workflows, we need to check the "Skip source verification" option. This way the job is started without waiting for the file to be "writeable"
*) We need a pure ffmpeg workflow for this and we need to put one option before the "-i" option of ffmpeg: "-follow 1", see the file protocol documentation: https://ffmpeg.org/ffmpeg-protocols.html#file. Additionally/optionally we could influence the time that ffmpeg waits for new data using the rw_timeout option.
*) Unfortunately we cannot yet influence the internal ffastrans encoding commands, we need to use a commandline processor for encoding
*) for testing the whole thing, you can also simulate a growing file using the windows type command. "type INPUTFILE > OUTPUTFILE"

Practical example:
*) Create a new workflow with a commandline processor and this content:

Code: Select all

"%s_ffmpeg%" -follow 1 -i "%s_source%" c:\temp\output.mxf -y

*) Simulate a growing file (limit speed to realtime) using the following commandline:

Code: Select all

ffmpeg -re -f lavfi -i mandelbrot  c:\temp\growing.mxf

*) Submit the growing file (c:\temp\growing.mxf) to the commandline processor

This way you should have successfully simulated a growing file workflow. Now you can alter the workflow, add a monitor node, check "skip source verification" and add your encoding settings and the correct output path to the ffmpeg command in the cmd processor.

Sidenote: if you record from SDI, you can potentially use ffmpeg directly for recording and transcoding in a single step

[Spectragr]

Thanks Εmcodem . I will try and get back. basically i want a recording in (mov or mp4) to be converted to mxf at the time of recording. I will get this record from the shared folder.

[emcodem]

In the case of mov and mp4 (which is technically 99% identical), the encoder has to write them with "faststart option". This is often an "opt-in" option, means it is disabled by default. Background is that without faststart, the header is written at the end of the file. As the header contains all info that is neccessary for decoding (codec, height/width etc...), one needs to wait until the header is written. "Faststart" option means the header is written at the beginning of the file.

You should be able to test if faststart is enabled for your mp4/mov when just trying to play the growing file in VLC or transcode it with ffmpeg. If it is not enabled, you need to check back with the encoder settings if it supports this option. If not, things become very complex.

Looking forward for your feedback

[Spectragr]

HDCAM 50i

Code: Select all

ffmpeg -i input.mov -c:v mpeg2video -b:v 50000k -s 1920x1080 -flags +ildct+ilme -top 1 -r 50 -c:a pcm_s16le -ar 48000 -filter_complex "[0:a]channelsplit[l][r]" -map 0:v -map "[l]" -map "[r]" -y output.mxf

and while I set with -r 50 frames it outputs 25 frames

-faststart doesn't work it might be the version FFmpeg 6.1

[emcodem]

I guess this is the command that shall read the growing file?
If so, you miss the "-follow 1" option before "-i input.mov "
Also, the faststart option i mention exists as long as mp4 exists but is not anyhow related to the encoding command, it is something your recorder must do. If you can play the growing file with vlc you dont need to worry about it, if not you have to look out for something like faststart or "put header at start" or similar in your recorder.

There is no real 50i, this name is a misconcept (this would mean 100half fields per second), only 25i (50 half fields per second, you might want to replace -r 50 by -vf fps=25. Or maybe -r 25 works but i think -r does not framerate conversion actually, fps filter does.

Starting with your example:

Code: Select all

ffmpeg -follow 1 -i input.mov -c:v mpeg2video -b:v 50000k -s 1920x1080 -flags +ildct+ilme -top 1 -vf fps=25 -c:a pcm_s16le -ar 48000 -filter_complex "[0:a]channelsplit[l][r]" -map 0:v -map "[l]" -map "[r]" -y output.mxf

Note that we don't produce standard compliant XDCAM here, if you send your file to a broadcaster it would be rejected, we need more options for that but before going on it is better to check if the growing part actually works.

[Spectragr]

growing doesn't work. you are right about the frames . what should i add? doesn't work -follow 1

[emcodem]

does vlc play the file?

[Spectragr]

Yes, the image in vlc is a bit stuck.

[emcodem]

OK so if you are sure that VLC plays the file WHILE it is growing (but a little stuck)...
The debugging will be a little problematic now.

Can you test a very simple ffmpeg command with the growing file like:

ffmpeg -follow 1 -i "\\server\share\folder\growing.mp4" c:\temp\output.mp4

And if this fails, can you please post the full, uncut ffmpeg output here? A screenshot from the cmd box will do as well but better is if you just copy whatever ffmpeg writes into the console