Speedup VMAF Calculation

[len23]

Hi
Is it possible to speed up the VMAF calculation?
I assume that you are using ffmpeg for the calculation.

With the command n_threads=x the speed of the calculation can be increased.

Thank you very much for this wonderful program

Greetings
len

[emcodem]

Hi len,

welcome to the Forum and thank you for using FFAStrans!
Funny enough i stepped over this topic very recently and i sticked to SSIM because i just assumed vmaf might be slow by design. At the end of the day i learend that SSIM is a lot worse as i was already aware it is:

ssim_is_bad.jpg (149.2 KiB) Viewed 10882 times

SSIM: 0.946235481
ME: this cannot be the real ssim value, this must be a joke... The picture looks completely different
SSIM: yeah, you know that i don't calculate values about how the pictures differ from human perspective, right?
ME:

Now as you throw up the topic, i did some testing with VMAF and the results are amazing:
At work i got some older HP Z820 workstation with 24 xeon cores, the speed was around 16 fps for fullhd.
Now i played at home on a very recent Core i5 13600k and voila, the speed raised to 22 fps, i guess because of the processor having a higher turbo frequency.

Then i added n_threads=20 and the speed was raised to about 100fps

@admin should really look into this, it's an easy change and it makes vmaf actually useable.

@len23 let me know if you have more urgent need for the speedup, there is a pretty simple way to do it with a cmd processor.

[len23]

Hi emcodem,
Thank you for your feedback.
I hope the developers are looking into this request.
I am currently doing my VMAF calculation outside FFAStrans.
Furthermore, I would be very interested in your CMD process, this would be another step towards a complete automation of my encoding process.

[emcodem]

Sure, so let me give you this example:

emcodem_custom_vmaf.json

(2.62 KiB) Downloaded 331 times

This workflow saves the final vmaf value in a variable like %s_vmaf% and sets s_success to it so you see the result in the job monitor.

The cmd is built like this:

Code: Select all

"%s_ffmpeg%" -i "%s_source%" -i "%s_original_path%\%s_original_name%_1.%s_original_ext%" -filter_complex "[0:v]split=1[vmaf0];[1:v]split=1[vmaf1];[vmaf0][vmaf1]libvmaf=ts_sync_mode=nearest:n_threads=20" -an -hide_banner -nostdin -f null -

as you see the second -i (the file to compare) is built using the original filename but adds _1, so this would compare the files
C:\temp\test.mxf and c:\temp\test_1.mxf

Now there is a couple of options, besides we could specify the vmaf model (here i dont specify any model which means we use the fullhd living room default model), we can compare just a portion of the clips by inserting something like

Code: Select all

trim=start=74:end=148,

at the %HERE% Position in: [0:v]%HERE%split=1 and [1:v]%HERE%split=1

For Reference, the internally calculated command looks like this:

Code: Select all

"%s_ffmpeg%" -i SOURCEFILE -i COMPAREFILE -filter_complex "[0:v]trim=start=74:end=148,split=1[vmaf0];[1:v]trim=start=74:end=148,split=1[vmaf1];[vmaf0][vmaf1]libvmaf=use model='path=C\:/temp/FFAStrans1.4.0.1/processors/FFMpeg/vmaf_v0.6.1.json':log_path='c\:/.ffastrans_work_root/20210121-2008-2041-5578-dc1cf46d7048/20240404-0036-4036-0181-13152608ff67/1-15-15~240404003642543~2132~20240404-0036-1586-9e74-2d30d687ed2c~anal_videoq~vmaf.json':log_fmt=json:pool=mean:ts_sync_mode=nearest" -an -hide_banner -nostdin -f null -

While looking at the libvmaf filter documentation, we can safely remove some of the options, especially the model path because 0.6.1 seems to be default. If you need to specify the model path, you must take great care about escaping the doubledot as seen above "c\:/temp" and use forward slashes for the rest of the path. The log_path i believe we don't need at all. Instead i just parse the vmaf value from the executed ffmpeg output in a populate variables processor by applying a regex.

Not to mention, the real speedup would be gained by using GPU power, you can just use the cuda version libvmaf_cuda by downloading and use a nonfree version of ffmpeg with "nonfree" components (or build yourself), like from here: https://github.com/AnimMouse/ffmpeg-autobuild
In this case, you migh not want to replace the internal ffmpeg.exe from ffastrans but just keep your own copy in a separate path and replace the path to ffmpeg in the cmd processor to point to your own ffmpeg version.

Further resources:
In case you want a GUI for selecting and submitting 2 (original and to_compare) files more comfortable, let me know and i will come up with a corresponding example workflow that you can use from webinterface.

[len23]

Many thanks for a detailed explanation emcodem.

Unfortunately I did not manage to get regext to work in my tests yesterday. I get no output in the txt file.

Attached is my test workflow.

len_custom_vmaf.json

(3.88 KiB) Downloaded 349 times

Could you perhaps tell me what I am missing?

Greetings

[emcodem]

So there are multiple things to say.

First in your workflow, the cmd processor, i needed to uncheck "show console" to get it working at all. I guess we cannot show the output in the console and get the output in ffastrans at the same time. To still see the commandline output you can use webinterface to investigate the job log, this will be helpful for your experiments anyway because it is designed to support you while building and debugging workflows.

Second, the regex of my example workflow can be replaced by this one to make it more robust:

Code: Select all

$regext("%s_stdout%","VMAF score: (\d+\.\d+)")

Third, in your cmd processor, you should specify the full path to ffmpeg. What you have is just "ffmpeg". Either replace this word by %s_ffmpeg% to work with the ffastrans ffmpeg or specify the full to your personal ffmpeg.exe, otherwise it will not work (at least not when running as service).

Fourth, if it makes your life easier, you can just omit the cmd /C "" part in the cmd processor, for ffmpeg commands it will work without this.

[admin]

Hi len23,

I can confirm that VMAF speedup will be featured in FFAStrans 1.4.1. Thanks for pointing out!

-steinar