Speedup VMAF Calculation

Use this forum to request features. Maybe you'll get lucky!
Post Reply
len23
Posts: 3
Joined: Sun Jan 28, 2024 5:43 pm

Speedup VMAF Calculation

Post by len23 »

Hi
Is it possible to speed up the VMAF calculation?
I assume that you are using ffmpeg for the calculation.

With the command n_threads=x the speed of the calculation can be increased.

Thank you very much for this wonderful program

Greetings
len
emcodem
Posts: 1686
Joined: Wed Sep 19, 2018 8:11 am

Re: Speedup VMAF Calculation

Post by emcodem »

Hi len,

welcome to the Forum and thank you for using FFAStrans!
Funny enough i stepped over this topic very recently and i sticked to SSIM because i just assumed vmaf might be slow by design. At the end of the day i learend that SSIM is a lot worse as i was already aware it is:

ssim_is_bad.jpg
ssim_is_bad.jpg (149.2 KiB) Viewed 2609 times
SSIM: 0.946235481
ME: this cannot be the real ssim value, this must be a joke... :? The picture looks completely different
SSIM: yeah, you know that i don't calculate values about how the pictures differ from human perspective, right?
ME: :roll:

Now as you throw up the topic, i did some testing with VMAF and the results are amazing:
At work i got some older HP Z820 workstation with 24 xeon cores, the speed was around 16 fps for fullhd.
Now i played at home on a very recent Core i5 13600k and voila, the speed raised to 22 fps, i guess because of the processor having a higher turbo frequency.

Then i added n_threads=20 and the speed was raised to about 100fps

@admin should really look into this, it's an easy change and it makes vmaf actually useable.

@len23 let me know if you have more urgent need for the speedup, there is a pretty simple way to do it with a cmd processor.
emcodem, wrapping since 2009 you got the rhyme?
len23
Posts: 3
Joined: Sun Jan 28, 2024 5:43 pm

Re: Speedup VMAF Calculation

Post by len23 »

Hi emcodem,
Thank you for your feedback.
I hope the developers are looking into this request.
I am currently doing my VMAF calculation outside FFAStrans.
Furthermore, I would be very interested in your CMD process, this would be another step towards a complete automation of my encoding process.
emcodem
Posts: 1686
Joined: Wed Sep 19, 2018 8:11 am

Re: Speedup VMAF Calculation

Post by emcodem »

Sure, so let me give you this example:
emcodem_custom_vmaf.json
(2.62 KiB) Downloaded 69 times
This workflow saves the final vmaf value in a variable like %s_vmaf% and sets s_success to it so you see the result in the job monitor.

The cmd is built like this:

Code: Select all

"%s_ffmpeg%" -i "%s_source%" -i "%s_original_path%\%s_original_name%_1.%s_original_ext%" -filter_complex "[0:v]split=1[vmaf0];[1:v]split=1[vmaf1];[vmaf0][vmaf1]libvmaf=ts_sync_mode=nearest:n_threads=20" -an -hide_banner -nostdin -f null -
as you see the second -i (the file to compare) is built using the original filename but adds _1, so this would compare the files
C:\temp\test.mxf and c:\temp\test_1.mxf

Now there is a couple of options, besides we could specify the vmaf model (here i dont specify any model which means we use the fullhd living room default model), we can compare just a portion of the clips by inserting something like

Code: Select all

trim=start=74:end=148,
at the %HERE% Position in: [0:v]%HERE%split=1 and [1:v]%HERE%split=1

For Reference, the internally calculated command looks like this:

Code: Select all

"%s_ffmpeg%" -i SOURCEFILE -i COMPAREFILE -filter_complex "[0:v]trim=start=74:end=148,split=1[vmaf0];[1:v]trim=start=74:end=148,split=1[vmaf1];[vmaf0][vmaf1]libvmaf=use model='path=C\:/temp/FFAStrans1.4.0.1/processors/FFMpeg/vmaf_v0.6.1.json':log_path='c\:/.ffastrans_work_root/20210121-2008-2041-5578-dc1cf46d7048/20240404-0036-4036-0181-13152608ff67/1-15-15~240404003642543~2132~20240404-0036-1586-9e74-2d30d687ed2c~anal_videoq~vmaf.json':log_fmt=json:pool=mean:ts_sync_mode=nearest" -an -hide_banner -nostdin -f null -
While looking at the libvmaf filter documentation, we can safely remove some of the options, especially the model path because 0.6.1 seems to be default. If you need to specify the model path, you must take great care about escaping the doubledot as seen above "c\:/temp" and use forward slashes for the rest of the path. The log_path i believe we don't need at all. Instead i just parse the vmaf value from the executed ffmpeg output in a populate variables processor by applying a regex.

Not to mention, the real speedup would be gained by using GPU power, you can just use the cuda version libvmaf_cuda by downloading and use a nonfree version of ffmpeg with "nonfree" components (or build yourself), like from here: https://github.com/AnimMouse/ffmpeg-autobuild
In this case, you migh not want to replace the internal ffmpeg.exe from ffastrans but just keep your own copy in a separate path and replace the path to ffmpeg in the cmd processor to point to your own ffmpeg version.

Further resources:
In case you want a GUI for selecting and submitting 2 (original and to_compare) files more comfortable, let me know and i will come up with a corresponding example workflow that you can use from webinterface.
emcodem, wrapping since 2009 you got the rhyme?
len23
Posts: 3
Joined: Sun Jan 28, 2024 5:43 pm

Re: Speedup VMAF Calculation

Post by len23 »

Many thanks for a detailed explanation emcodem.

Unfortunately I did not manage to get regext to work in my tests yesterday. I get no output in the txt file.

Attached is my test workflow.
len_custom_vmaf.json
(3.88 KiB) Downloaded 104 times
Could you perhaps tell me what I am missing?

Greetings
emcodem
Posts: 1686
Joined: Wed Sep 19, 2018 8:11 am

Re: Speedup VMAF Calculation

Post by emcodem »

So there are multiple things to say.

First in your workflow, the cmd processor, i needed to uncheck "show console" to get it working at all. I guess we cannot show the output in the console and get the output in ffastrans at the same time. To still see the commandline output you can use webinterface to investigate the job log, this will be helpful for your experiments anyway because it is designed to support you while building and debugging workflows.

Second, the regex of my example workflow can be replaced by this one to make it more robust:

Code: Select all

$regext("%s_stdout%","VMAF score: (\d+\.\d+)")
Third, in your cmd processor, you should specify the full path to ffmpeg. What you have is just "ffmpeg". Either replace this word by %s_ffmpeg% to work with the ffastrans ffmpeg or specify the full to your personal ffmpeg.exe, otherwise it will not work (at least not when running as service).

Fourth, if it makes your life easier, you can just omit the cmd /C "" part in the cmd processor, for ffmpeg commands it will work without this.
emcodem, wrapping since 2009 you got the rhyme?
admin
Site Admin
Posts: 1670
Joined: Sat Feb 08, 2014 10:39 pm

Re: Speedup VMAF Calculation

Post by admin »

Hi len23,

I can confirm that VMAF speedup will be featured in FFAStrans 1.4.1. Thanks for pointing out! :-)

-steinar
Post Reply