fifo (watch folder with auto appending files) to stream

[sdm]

I am looking for solution for restream files from FIFO ( record 24/7 from SDI to MXF files ) to the HLS stream
files appears 24/7 and auto refresh in folder named in format YYYY_MM_DD_HH_MM
for ffmpeg exist follow solution
https://trac.ffmpeg.org/wiki/Concatenate#demuxer ( topic: Automatically appending to the list file )
but i am looking for solution for Windows

[emcodem]

Hi sdm,
big problem but little description you provide.
If you want to use the concat filter, how would you decide which chunks from your recording system are "concatted"?

[sdm]

My case with FIFO is:
path to main FIFO folder ( constant path )
FIFO clock folder in format: YYYY_MM_DD_HH_MM ( that mean new name for FIFO folder every hour )
files in FIFO folder in format: YYYY_MM_DD_HH_MM_SS.MXF ( at same time exist temp file with tmp extension that renamed to MXF after chunk complite ( reach 30 sec duration ) )
I see follow solutions for restreaming FIFO:
Streaming most resent file ( my case ), that mean that handler must looking on main FIFO folder for last file with only MXF extension ( lastwritetime ) for miss tmp file ( that incomplite ), at same time handler must create fifo file that start streaming with ffmpeg to HLS, and copy every new file to this pipe ( in example by link above, pipe created for playlist, but i think creation pipe for video files is more simple, because theoreticaly ffmpeg must working with pipe like with simple growing file )

[emcodem]

Thanks for some more insights!
Sorry i forgot to welcome you! - WELCOME to the forum and thank you for using ffastrans.

So if i understand correct, you want to output a HLS live stream from the recorded chunks, is that correct?
Basically, ffastrans cannot really help you here a lot as we concentrate on non-live stuff. - I see no chance for a feature request here as the base of ffastrans does not support any live stuff.

Lucky you, i was working on that topic already for some weeks, so here is some of my knowledge:
The only way that i am aware of dynamically adding input files to a single running instance of ffmpeg is to feed a m3u8 playlist.
Try it out, create a text file like this (C:\temp\mxf.m3u8 ):

Code: Select all

#EXTM3U
#EXT-X-VERSION:3
#EXT-X-MEDIA-SEQUENCE:0
#EXTINF:30,
c:\temp\1.mxf

Then start an ffmpeg instance, e.g.

Code: Select all

ffmpeg -allowed_extensions ALL -i C:\temp\mxf.m3u8 -c:v libx264 -crf 21 -preset veryfast     -c:a aac -b:a 128k -ac 2   -f hls -hls_playlist_type event -hls_list_size 6 -hls_flags delete_segments  c:\temp\LIVE\stream.m3u8

Then, at the end of your mxf.m3u8, append the next mxf segment,

Code: Select all

#EXTINF:30,
c:\temp\2.mxf

Your mxf.m3u8 now looks like this:

Code: Select all

#EXTM3U
#EXT-X-VERSION:3
#EXT-X-MEDIA-SEQUENCE:0
#EXTINF:30,
c:\temp\1.mxf
#EXTINF:30,
c:\temp\2.mxf

You will notice, that the encoding ffmpeg instance picks up the new file and continues transcoding the live stream.
Using ffastrans, you could watch for new *.mxf chunks and append each found chunk to the mxf.m3u8 playlist using the text processor

Anyway, i never got it working this way, it is just too much effort to solve all problems, e.g. your audio WILL get async, you need to take care about missing or defect clips from the recording systems and much more.
If this is a commercial project, it will be best for you to search for ready to use solutions that you can buy instead of investing months and years on your own, this way the time to market is shorter and the final result is much more stable. E.g. did you talk to the guys from wowza?

[sdm]

I have tested your solution yesterday ( via powershell script ).
Now it slill look like tests, and not ready for production, but as conception it is working!!
Thank you for this idea.

[sdm]

Anyway, i never got it working this way, it is just too much effort to solve all problems, e.g. your audio WILL get async, you need to take care about missing or defect clips from the recording systems and much more.
If this is a commercial project, it will be best for you to search for ready to use solutions that you can buy instead of investing months and years on your own, this way the time to market is shorter and the final result is much more stable. E.g. did you talk to the guys from wowza?

Yes this is commercial project, and equipment manufacturer ( that MXF created ) ready to do this task on request, but this case is solution for users, as quick preview for materials, that they can use in editor ( in MXF format ), and ffmpeg solution is sufficient for my case.

[emcodem]

Hey sdm,
good to know it kind of worked for you, thanks for reporting back.
Do you really need to transcode to HLS or couldn't you let the users play the playlist with the mxf's too? - this way everything would be a lot more simple...

[sdm]

No, i want to play HLS via web browser, MXF format is 50 Mbit/sec, it's also too much for this reason, that why i need to transcode it to 264 or 265 format.

[emcodem]

As you dont really need much features of HLS (but just want a browser compatible format), you could and should try to get rid of the second ffmpeg instance that's long running process.
I am under the impression that you could just transcode to h264 .ts files and create a playlist for the clients as we did in mxf.m3u8; but specifying the .ts files instead of the mxf files.
This way you just need to transcode every 30s chunk to .ts and append the name to the playlist. Every night, you delete the playlist.
Something like that is what i would target. It is pretty important to get rid of this long running ffmpeg process if possible. Because it is hard to keep alive and hard to debug etc...