Dual GPU encoding

[llittleton]

I have been trying out using Rigaya's Nvencc:

https://github.com/rigaya/NVEnc/

Using this command line:

%comspec% /c "D:\NVEncC_5.11_x64\NVEncC64.exe -i "%s_source%" --fps 30000/1001 --output-res 1920x1080 -c hevc --cbr 10000 --lookahead 16 --aq --profile main10 --output-depth 10 --audio-codec aac --vpp-edgelevel --vpp-deband --video-tag hvc1 --cuda-schedule auto -o "%s_original_path%\%s_original_name%_HD.mp4""

The advantage is that it enables gpu decode and encode for HEVC 10bit, plus it load balances on both Nvidia P4000 Gpus!

XAVC-I UHD 60 H264 10 bit to UHD 30 HEVC 10 bit runs at 30 fps (no gpu decode), can run 2 - 3 sessions.
UHD 30 HEVC 10bit to HD 30 HEVC 10bit runs at 170 fps (the above command line). can run 4 - 6 sessions.
HD 30 HEVC 10bit to 720p runs at 300 fps, can run 8 - 12 sessions.

Good so far! It would be of great benefit to be able to specify the max jobs per workflow.

I have not figured out how to limit the number of audio channels with this workflow, all 8 audio channels are passed to the output videos.
I really only need to keep the first two audio channels. Rigaya's documentation is not clear on audio options.

Thanks again,

Lawrence

[emcodem]

hey @littleton

Here is a small hack that allows you to control the jobs per workflow.
viewtopic.php?f=5&t=875&p=5384&hilit=li ... face#p5387

As a result of this thread, we already experimented with smarter queue controls and are making our way slowly. There are many things to take into consideration for smart que management, it could take a while...

Regarding the audio channel config, unfortunately i cannot help you, never heared of the tool you are using. Maybe you can port it to ffmpeg at some time, then it will be easy to help you Also you could probably pipe the output of your tool to ffmpeg and take care about the audios there... or run ffmpeg with -c:v copy after your tool finished the encoding, taking care about the audios only.

[llittleton]

@encodem

Thank you for your reply. I will look at the workflow hack and will try your suggestion of an ffmpeg node to strip out the audio channels.

Lawrence

[llittleton]

Follow up on dual gpu, I am using this from above:

cmd /C ""C:\Program Files\NVIDIA Corporation\NVSMI\nvidia-smi.exe" --query-gpu=encoder.stats.sessionCount --format=csv,noheader -i 0"

Works fine if jobs come in not at the same time like from a watch folder. But if I load in 3 or more files into workflow, they all go to the first gpu because they all load in simultaneously. So for manual submissions I need to load in 2, then 2 more. Any ideas how to be able to load in a folder full of files, maybe have a delay between when each subsequent file loads in so that the gpu actually has time to load up?

Thanks again,

Lawrence

[admin]

Hi littleton,

I think I might have something for you, provided you run at least FFAStrans 1.0 or higher:
Locate the file FFAStrans\Processors\db\configs\ffastrans.json then open it in a text editor. Find the string "abs_min": 250 and change the value to something like 1500 and save again. What this will tell FFAStrans is to wait for 1500 milliseconds (1.5 seconds) before initiating next job ticket. If it does not help your situation try an even higher value, like 3000, or even 5000. This SHOULD be enough but I don't really know how long it takes for your GPU to register the sessions so you need to experiment.

Please try and report back.

PS: After changing the "abs_min" value, please wait for about 30 seconds for the changes to take effect on the system. No reboot or restart of FFAStrans is needed.

-steinar

[llittleton]

steinar,

This had no effect, even when set to 30000 ms. I see the jobs load in very quickly, and always to gpu0.

I am comparing FFASTrans to Root 6 Content Agent ($25k) currently. So far you are the winner in terms of
speed and reliability.

Thank you again for your excellent software!

Lawrence

[emcodem]

Hey @llittleton

Wow actually a pretty nasty problem you have there, not easy to solve. Exactly the kind of problem that i like most, thank you for this question.

Here my attempt. I generated a very simple plugin processor that waits for a given file to be the oldest in it's directory.

In order to use the workflow below, you must install the "Wait until File is oldest" processor:
https://www.ffastrans.com/wiki/doku.php ... processors

Here the workflow:

emcodem_control_paralell_jobstarts.json

(3.58 KiB) Downloaded 384 times

The idea is that each job writes a text file on startup. The wait for oldest then waits until the "current" job was the one who wrote the oldest file. It then sleeps a few seconds in order to give ffmpg time to wind up nvidia and only after that sleep, we delete the current job's file from the quorum directory.

Of course you must ensure that there are no other files in the quorum location (i used c:\temp\quorum), like thumbs.db or such.

Let me know if it works for you!

[admin]

Hey llittleton,

I'm sorry to have mislead you but it does not seem changing the "abs_min" parameter works in 1.1.0 My stupid mistake!
It's however fixed in the upcoming release and it really should make a difference in your scenario. The next release has been delayed but wiill hopefully be out Sunday.

-steinar

[llittleton]

encodem and steinar,

Thanks for the feedback! We are down because of the Texas weather, should be back by Monday. I will wait for the update to test.
If the update is delayed I will try encodem's fix.

thanks again,

Lawrence

[momocampo]

Hi Lawrence,
Don't worry, the update 1.2 will be available on Sunday the latest.
B.