Numa nodes

Questions and answers on how to get the most out of FFAStrans
Post Reply
PoznanskaPyra
Posts: 7
Joined: Mon Jan 04, 2021 3:02 pm

Numa nodes

Post by PoznanskaPyra »

Hi

When ffastrans transcoding few files on same time (in one workflow), then uses only one NUMA node.
Is possible to run one video transcode on numa node 1, and run next video transcode on numa node 2?
Attachments
trcode.PNG
trcode.PNG (19.78 KiB) Viewed 3626 times
numa2.PNG
numa2.PNG (14.76 KiB) Viewed 3627 times
numa1.PNG
numa1.PNG (12.27 KiB) Viewed 3627 times
emcodem
Posts: 1811
Joined: Wed Sep 19, 2018 8:11 am

Re: Numa nodes

Post by emcodem »

Hubert,
you hit me very deep with that question, i just love it (because i know about NUMA nodes but i dont have much experience with it) and i promise to find a solution together with you as long as you keep on delivering all infos we need. Depending on the complexity, hopefully the solution joins the ffastrans source code later on if needed.
At least i know from another transcoder that they imlepmented "something" about it and charge a license for it, so lets get out if its worth any money they charge.

So, we had an internal talk and it looks like steinar alredy has seen full cpu utilisation on a 24 core system (48 core with HT - which hits the 36 core limit for NUMA, at least as far as i know).
Our expectation would be that whenever the application (exe_manager) spawns a new child process, the OS will smartly manage which processor group is assigned to it. Your problem description points us to a direction where this is just not the case and all child processess of exe_manager just share the same processor group as the parent process is assigned to.

You know, as usual: first step in problem solving is to reproduce the problem...

1) Therefore, the first thing we need is your workflow exported and uploaded here. Of course, if you can simplify your workflow to e.g. a single encoding node where you just submit 5 files and you got stuck at utilisation bound to a single NUMA node, it would be tremendious!

2) About your system: i assume it is not a vm Guest but directly on hardware installed windows, right?

3) Just to make sure, you have Hyperthreading enabled, right?

4) what OS version are you running?

5) Why you dont have webinterface installed? (:D)
emcodem, wrapping since 2009 you got the rhyme?
emcodem
Posts: 1811
Joined: Wed Sep 19, 2018 8:11 am

Re: Numa nodes

Post by emcodem »

@PoznanskaPyra
luckily, i got some new servers with 32 Cores now and what i found out is that your problem might be less connected to NUMA Nodes but to processor groups instead. In my HP Server DL380 Gen10, i found some NUMA related option that when set to "Cluster", it would force windows to Split my 64 logical cores into 2 groups while i expected to have only 1 group.
So, your goal is always to have only 1 processor group (unless you run with more than 64 cores, in that case let's continue the discussion). You can e.g. simply disable Hyper-Threading as there is no benefit from it for transcoding anyway.
emcodem, wrapping since 2009 you got the rhyme?
PoznanskaPyra
Posts: 7
Joined: Mon Jan 04, 2021 3:02 pm

Re: Numa nodes

Post by PoznanskaPyra »

Sorry. I have lot of jobs and i forgot about this thread :/ At this moment ffastrans is base transcoder, it has 5-6 jobs in one time. CPUs is allways loaded to 100% :)
Post Reply