Custom ffmpeg - XDCAMHD 1080i50

[Tomical]

Hello,

I would like to use "custom ffmpeg" to burn subtitles and transcode my file to XDCAMHD 1080i50. I'm almost there but I'm missing command lines for example to have the right color space (BT. 709), interlaced ==> top field first, etc.
Here are the properties I would like to have for my output file:

Video
ID : 512
Format : MPEG Video
Commercial name : XDCAM HD422
Format version : Version 2
Format profile : 4:2:2@High
Format settings : CustomMatrix / BVOP
Format settings, BVOP : Yes
Format settings, Matrix : Custom
Format settings, GOP : M=3, N=12
Format settings, picture structure : Frame
Format settings, wrapping mode : Frame
Codec ID : 0D01030102046001-0401020201040300
Duration : 3 min 55 s
Bit rate mode : Constant
Bit rate : 50.0 Mb/s
Width : 1 920 pixels
Height : 1 080 pixels
Display aspect ratio : 16:9
Frame rate : 25.000 FPS
Standard : Component
Color space : YUV
Chroma subsampling : 4:2:2
Bit depth : 8 bits
Scan type : Interlaced
Scan order : Top Field First
Compression mode : Lossy
Bits/(Pixel*Frame) : 0.965
Time code of first frame : 00:00:00:00
Time code source : Group of pictures header
Stream size : 1.37 GiB (97%)
Color range : Limited
Color primaries : BT.709
Transfer characteristics : BT.709

Actually in my custom ffmpeg :
Video codec : mpeg2video Audio codec : pcm_s16le
Bitrate : 50000 Bitrate : 768

Options =
-map 0:0 -map 0:1 -map 0:2 -vf "subtitles='c\:\\subs\\%s_original_name%.srt':force_style='FontName=Arial,fontsize=16,Alignment=2,outlinecolour=&h80000000,MarginL=5,MarginV=25'"


If someone has the good command lines, it would be great !!!

Thanks

[emcodem]

Aye @Tomical
welcome to the forum and thank you for using ffastrans!

Thats a good question but as i see it, what you want to do is to copy the command that is generated by the xdcamhd processor. To get the full command, follow this thread:
viewtopic.php?f=5&t=849&p=4486&hilit=beginners#p3641

This tutorial was written for version 9.4, these days it is best if you just use the webinterface to access the logs because the logs are in JSON format now...
Note that lots of parameters are calculated dynamically and depend on the source file, i am sure you figure that out on your own

[admin]

Hi Tomical,

These encoding parameters should get you going:

-c:v mpeg2video -r 25 -pix_fmt yuv422p -aspect 16:9 -intra_vlc 1 -b:v 50000000 -minrate 50000000 -maxrate 50000000 -bufsize 17825792 -rc_init_occupancy 17825792 -bf 2 -non_linear_quant 1 -color_primaries bt709 -color_trc bt709 -colorspace bt709 -seq_disp_ext 1 -video_format component -color_range 1 -chroma_sample_location topleft -signal_standard 4 -dc 8 -qmin 3 -qmax 28 -g 12 -field_order tt -top 1 -flags +ildct+ilme -alternate_scan 1 -c:a pcm_s24le -ar:a 48000

Note that this says nothing about audio mapping or any kind of filter processing. These are just the parameters needed to make a compilant(ish) XDCAM-HD 422 file.

-steinar

[Tomical]

Hello,

Thank you very much, everything works !

Regards,
Thomas

[zabalex]

Hello Steinar,
Your answer to Tomical about 1.5 years ago on how to encode XDCAMHD video just literally saved my job earlier today, and my employer's reputation with one of its new clients.

In short, I was breaking my head for the past 2 days on how to make XDCAMHD string to appear on my ffmpeg processed video. Input, and client's specification always showed it as 'codec commercial name', but my output produced nothing. Until I've found your post from above.
I just made some tiny adjustments, such as I needed M=3/N=15 instead of 12, that is, for my use-case, but everything else fixed the issue.

I understand most of the parameters in your input, except just a few, is there some documentation where I could read more about it, or you would care to elaborate, please?

-intra_vlc ?
-rc_init_occupancy -- /* found this description: -rc_init_occupancy E..V…. number of bits which should be loaded into the rc buffer before decoding starts (from INT_MIN to INT_MAX) (default 0)*/
-bf -- /* set maximum number of B frames between non-B-frames (from -1 to INT_MAX) (default 0) */
-non_linear_quant -- /* Use nonlinear quantizer. (from 0 to 1) (default 0) */ <-- I'm still confused on the meaning of this
seq_disp_ext ?
-video_format ?
-signal_standard 4
-dc 8
-qmin 3 /*this one I understand since I could make the connection*/
-qmax 28
-g 12 /*same as above*/

and probably the most important is, which one actually allowed to display XDCAM HD422 on the output?

Again, thank you so much,

[emcodem]

Hey @zabalex
welcome to the forum and thank you for (maybe) using FFAStrans
Very nice questions, i like it just i like your avatar! I'll leave the parameters part for @admin to answer, just going for your question about what makes Mediainfo to display format_commercial XDCAMHD422, it is actually pretty funny and trivial, read here please: viewtopic.php?f=5&t=1220&p=6462&hilit=m ... code#p6462

Don't forget that even if the mpeg2 essence stream is 100% compliant, the ffmpeg mxf container does not fulfill XDCAMHD requirements, it misses some metadata entries in the header and therefore the files cannot be transferred to Sony professional disk. Read here: https://trac.ffmpeg.org/ticket/5097
So when going for the full thing, you should rewrap the output from ffmpeg using bmx. One can do that inline, so without time loss but i dont have an example commandline at hand, lemme know if you need it.

Also, please know that we just recently detected a massive quality issue with the qmin and qmax settings you copied above, and we are not yet done finding out the final solution for this problem. The current state on this is that -qmin 5 -qmax 23 seem to be better settings than -qmin 3 -qmax 28.
See here: viewtopic.php?f=4&t=1261&hilit=qmin#p7070

[zabalex]

Thank you so much for such a prompt response!

About qmin and qmax, I just needed to match the source video that was given to me by the broadcaster, and as you can see I'm almost 100% fit. The trick of mine, at this current stage, at least, is that they are not going to check frame-by-frame quality of images, but do a simple format-comparison, if matches I pass, if not, they auto-reject. So, I'm ok with that for now.

On a separate note, you have no idea how happy I am to come across your website and resources, in the past year or so I've been struggling finding all the necessary arguments for my ffmpeg encoder, and every argument beyond the basic ones are like pulling teeth out, with struggle.
I was writing my own python-based video format detection library with ffprobe/mediainfo cli, and outputting the arguments for to re-encode those. But it is hard for one person to summarize all the possible permutations of codecs and all the formats, and then output them as a string values to reuse. I just hope that if I ran into some issue of finding some argument like the XDCam, I have another wonderful team to approach and hopefully without much trouble on your part, to give me some clues

To your question of maybe using FFAStrans, I was looking for documentation on how to launch the installer on MacOS, but couldn't find any references to it. Do you even do a build on Mac? If not, maybe I could help at some point of time to also propagate to other OSs? Meanwhile, is the code available on github so I could play with my mac build?

Screen Shot 2021-10-14 at 11.36.47 AM.png (138.57 KiB) Viewed 9616 times

[admin]

Hi zabalex,

I'm sorry but FFAStrans is Windows only and will probably stay that way for an unforeseeable future. But we're anyway happy our project helped you find your way in the ffmpeg and the parameters jungle
I'm not an expert myself in all the ins and outs of the XDCAM-HD standard and most parameters have come along for a long period of time. That is, they have changed a few times based on feed back and new knowledge. But most parameters are taken directly from the deprecated ffmbc project and standards documentation I was able to translate into ffmpeg parameters. History have showed that they make quite compatible files in most scenarios.
When it comes to the specific parameters I can only try and give some kind of one-liners but you should really use our trusted google do find out what they really are. I guess books can be written about it.


-intra_vlc = (1) slight improved quality for higher bit rates and needed for XDCAM-HD compatibility.
-rc_init_occupancy = (17825792) Number of bits which should be loaded into the rc buffer before decoding starts
-bf = (2) number of b-frames for every p/i frames
-non_linear_quant = (1) Quantizer model that keeps more of the human visible details
seq_disp_ext = (1) always write display metrics and color metadata
-video_format = (component) write video format metadata
-signal_standard = (4) set video signaling to HD
-dc = (8) bit precision for quantized DC coefficients of intra-coded blocks
-qmax 28 = (lowest) quantizer "quality"

Not sure how much this helps but, just ask if you have some specific questions. Our FranceBB can probably give you some sexy details

-steinar

[emcodem]

I was writing my own python-based video format detection library with ffprobe/mediainfo cli, and outputting the arguments for to re-encode those.

There is a lot to say about that. Basically you might re-do what a lot of other guys already tried to do but they gave up because it was much cheaper to buy some existing solution compared to do it your own with ffmpeg. But honestly, when i first learned about ffmpeg (about 15 years ago), i thought that it is impossible to tame because the commandline parameters might change very frequently, so i now think as long as someone knows ffmpeg good enough, it is totally worth to start such a project.

If not, maybe I could help at some point of time to also propagate to other OSs?

I want you in the team
FFAStrans is written in Autoit, which is a 100% native windows scripting language, there is no way besides using Wine to use it on other platforms. So after all we do not have the option to support other OS than windows. What actually happens when we "compile" the source code is the same as in any other scripting language: the interpreter executeable itself is being packed along with the script code into a single file. This is the main reason why we are detected as a virus pretty often... Autoit is just used too little overall on this wold.
Anyway, rewriting it in another language will not happen. Honestly, it is still my long-term goal to make ffastrans cross-platform compatible.
There are other options to use ffastrans cross-platform without rewriting the whole code tough but this might be too much to write in this thread.

[zabalex]

Hi zabalex,

Not sure how much this helps but, just ask if you have some specific questions. Our FranceBB can probably give you some sexy details

-steinar

Thanks guys for being so kind to one of your newest members That was a lot of details, and I get it, at least 3 of the above I already digested into my python logic.
What was really interesting, and surprising the most is, after plugging in one parameter after the other, one by one, and monitoring the difference in my video output format, I was willing to bet it would be either seq_disp_ext or -video_format that actually made the XDCAM HD422 to appear, those two I was unfamiliar with until last night. But instead it was:
-minrate 50000000 \
-maxrate 50000000

I had those values sat as:
-b:v 49915464 -maxrate 49915464 -minrate 49915464


Why 49915464? Because this is what I read from parsing the mediainfo cli->json:

Screen Shot 2021-10-14 at 6.55.50 PM.png (44.06 KiB) Viewed 9587 times

In any case, simply changing to 50mb from your original last year's post made the trick. Just that alone.

I'm still a bit confused, what is 49915464? Where does ffprobe grab this data from? <-- this is where I need to rethink my logic. I executed my encoding process twice, so 2 times I was cutting on a number in the bitrate. MediaInfo does read this BitRate value correctly, and this is the change I already implemented today.

In any case, I've learned a lot in the past 24 hours, and arguments like qmax, qmin and '-g' puzzled me for sometimes now, just by accidentally running into your forum, I know how to manipulate those now and make my output a lot more precise.

I'm super happy, and thanks again to all of you for your warm welcome!
Keep in touch, as I will be visiting often, hopefully with some contribution to your board as well

BTW, I'll consider Windows route, your process is intriguing for my purposes. But, if you'd ever need some extra vote to plot your code to other OSs, I'm very much interested in that. Mac is what I personally use, but the biggest gain I'd see is from any linux deployment, maybe even a commercial-side for your FFAStrans company.

Screen Shot 2021-10-14 at 7.30.10 PM.png (24.46 KiB) Viewed 9586 times