FFMpeg encoding CPU limit for new jobs

[veks]

Hi all!

I see a bottleneck in my workflow, or FFAStrans as such.
Everything works fine, but when FFAStrans takes 1 file to transcode, and when it gets to multiple nodes for encoding multiprofile (6 profiles) it takes randomly let's say 3 jobs (3 jobs set as maximum), and then starts transcoding 3 profiles at the same time hogging a CPU to 99%.

Problem is that if I set a job to 1, some of the condition nodes won't fire, and then FFAStrans would just wait and wait, and won't do nothing.

What if you could set the number of simultaneous ENCODING FFMpeg jobs and CPU percentage when to take new ENCODING jobs?
I mean, it might be a problem with my workflow, some example of a workflow with at least 6 multi profile h264 transcoding would be also great.

If I missed an topic, please move it to somewhere else.

Thanks a lot!

EDIT 1: or ability to set maximum threads per encoding job. It's a must feature. I see no way to adding this option, only if using a totally custom FFMpeg command.

[emcodem]

Hey veks!
i am sure that @admin has something to say about that; personally i don't like the CPU limit feature at all, it should be set in transcoding slots only than in CPU percentage but that is not a matter of discussion.
However, i have an important question for you: do i understand correct that you are using a single machine for your encodings and your ultimate goal is to utilize it's resources as best as possible? - additionally you cannot affort to just add enough System Resources or Servers to your Farm, correct?

Thanks!
-emcodem

[veks]

Hey veks!
i am sure that @admin has something to say about that; personally i don't like the CPU limit feature at all, it should be set in transcoding slots only than in CPU percentage but that is not a matter of discussion.
However, i have an important question for you: do i understand correct that you are using a single machine for your encodings and your ultimate goal is to utilize it's resources as best as possible? - additionally you cannot affort to just add enough System Resources or Servers to your Farm, correct?

Thanks!
-emcodem

We're using an virtual machine with Xeon 6130 CPU.
But yeah, we would like to add another nodes, servers into farm.

[admin]

Hi veks,

First I just want to be clear on what you problem is. So you have the 99% CPU hogging, and the nodes not starting when you have limited simultaneous jobs to 1. Now, the latter worries me the most. Could you please share some details, or better yet, export and post your workflow?

So regaring CPU control... utilizing all the CPU power that is is sort of what encoders should do. I can see this is a challenge if you're using the same host for other tasks but the nature of this program is to run on separate server dedicated to transcoding. If you want to do other stuff, setting max jobs to 1 should do but not if there is a bug of course preventing FFAStrans from working correctly with this setting. So we need to address that first, before adding ad-hoc CPU control.

-steinar