ffmpeg.exe is set to High base priority

[no_connection]

When converting to ProRes (have not ran anything else to test) ffmpeg.exe is set to high base priority instead of normal or preferably low.
This breaks anything else running on the machine including windows as it's being starved of any resource making navigating even task manager to set correct base priority a hassle.

ffmpeg need to be run at normal or below for FFAStrans to even function at all.

[emcodem]

i have a good feeling about you no_connection, keep it coming

[FranceBB]

The process priority follows the workflow priority, to be fair.
Try setting your workflow to Priority 0 in the Workflow Manager when you right click on it and you'll see in Windows that the process called "ffmpeg.exe" is gonna have priority Low once the job begins. This will avoid the host to be resource starved.

p.s if it doesn't, then it's a bug, but I'm like 95% sure it was one of the "features" I checked before the last public release made by Grandmaster Steinar

[no_connection]

Having process priority follow workflow priority above normal makes less sense for the reason I mentioned. It should just be queue priority above normal (imo). With maybe going between normal, below normal and low just to have the system be more stable and select how low you want go. Having high base priority will probably mess up watch folder and other similar tasks as well as webgui and status. Else someone is going to add some heavy jobs set to high and wonder why nothing is responding

I just opened up a fresh FFAStrans and made a quick workflow to convert to ProRes since Resolve don't support any file formats at all.

It was set at normal before and changing it to lowest makes no difference. ffmpeg.exe still launches with high base priority. So it looks like a bug there.

[emcodem]

we need @admin here.
Sure it is like @FranceBB says (ffmpeg prio should follow workflow prio) but there are exceptions like manual submits override the priority and set it to high. So i guess @no_connection did this for testing. My guess is when you @no_connection use api (e.g. webinterface) or watchfolder submit, the process prio stuff works as expected.
Neverthless i'd like to use this as a starting point for more discussion about process prios because it is as @no_connection said: ffastrans always has to have the absolute highest priority, otherwise it just don't work. In my head, the prio should be always on the foreground windows form (if any), after that ffastrans other's stuff like exe_manager and only after that comes all the internal stuff like the analyzing processes and as very last thing there comes ffmpeg.exe.
Anyway, steinar will hopefully post some words about the intentions and enlighten our minds!

[no_connection]

@emcodem You are correct, I was manually submitting the files, which would explain the high priority then. Which is bad in this case.
I suppose that if you never use anything above normal in regular use then it's fine, use caution with higher. But then manually submitted jobs has to follow the same setting.

The only reason to set anything above normal in my opinion is video player and OBS and those type of real time applications that would stutter if they where not getting resources when needed right away instead of waiting.

I would argue that an application that intends to use 100% CPU like ffmpeg don't benefit at all by being set to more than low, yes maybe 1-10% if the resource it could fight back for at the expense of everything else. And base priority is not really the place to determine what jobs get more resources if two are running anyway. With some exception like one jobs that is more important is resource starved by for example network while the low prio job isn't, then you can have the network starved job be "ready" to receive resources right away while the low prio takes up any left over CPU cycles. That is a bit edge case tho.

Maybe have a check box with "override ffmpeg base priority to low" then we have best of both worlds.

[FranceBB]

we need @admin here.

He's on vacation (or "was" since it's the end of the week ), let's let him enjoy a bit of spare time xD

Sure it is like @FranceBB says (ffmpeg prio should follow workflow prio) but there are exceptions like manual submits override the priority and set it to high.

Yep, I double checked (it took me a bit of time 'cause I'm always on my messy branch xD) and the last publicly available version does indeed follow the workflow priority (as it should), so no bug at all.

@emcodem You are correct, I was manually submitting the files, which would explain the high priority then.

Yes, priorities go from 0 to 4 where 4 is the highest you can get, however this applies only to stuff like watchfolders.
IF you open the workflow manager and you submit a job manually, then it goes with a "hidden" level 5 priority.
This again is normal behavior and it's expected to work like this 'cause if you're "bold enough" to get into the Workflow Manager yourself and start a job manually, it means that either you want to test something or that you're really really really in a rush, hence the top priority above everything else. This is expected 'cause we decided the behavior long time ago together with Grandmaster 'cause if a user manually submits something it means that he wants the result immediately, so it's normal that FFAStrans takes the job and executes it immediately. So... yeah, that's why you were still seeing ffmpeg.exe with the highest priority in Windows too.

Which is bad in this case.
The only reason to set anything above normal in my opinion is video player and OBS and those type of real time applications that would stutter if they where not getting resources when needed right away instead of waiting.
I would argue that an application that intends to use 100% CPU like ffmpeg don't benefit at all by being set to more than low, yes maybe 1-10% if the resource it could fight back for at the expense of everything else.

Well, first things first, most of our users work in companies and have dedicated servers to run those kind of tasks. This means that they run on servers and not workstations or consumer laptop, which also means that whenever a standalone FFAStrans or indeed a farm takes a job, its only "duty" is to carry that job to the end without doing anything else in the background. So... nope, servers won't be recording stuff with OBS Studio, browsing the web with Chrome or anything else as they're... well... servers xD
Now, on to the priority stuff and why it matters for processes.
I feel partially responsible for this 'cause when it came to decide what to do, we decided this together with Grandmaster and it was actually something I wanted and tested and it's something that I use on a daily basis, especially since we launched our remastering of old tapes in 2020.
You see, companies might have huge farms with plenty of servers running all sorts of different kind of encodes. There are things with short turnaround (like for instance football highlights etc that need to be encoded and published) and other things that can wait a bit longer like normal contribution encodes for the productions or stuff that you have to encode for air-time in like 2-3 weeks time like movies etc and also stuff you may wanna run in the background using as little as possible like Remastering of old BetaCAM tapes, 1 inch Type C, U-Matic, you name it. If that's the case, simply setting the priority in the workflow manager will solve headaches 'cause not only it will assign different priorities to different tasks so that the tickets will be picked up and the jobs will be executed only if there are enough slots available so that stuff like priority 3 will go ahead of priority 1 jobs etc BUT (and bear with me on this) if you have like a priority 4 job that needs to be run and you have a priority 0 job that is already going, the exe_manager will PAUSE the priority 0 job, make the highest priority 4 job go through and finish and then UNPAUSE the priority 0 job which will slowly but surely reach the end. The way we do this ain't black magic, however is complicated and it does involve messing with Windows process priorities, so... there's a reason behind all this.

Anyway, this is to say that I'm not ignoring you and I'm not giving you a "hard no", on the contrary, I agree with emcodem and we will bring this up to Grandmaster, but keep in mind that we developed things this way 'cause there was a reason and it seemed to be working for lots of our user base, but yeah, we're open to new use-cases, after all, it's thanks to our users like you who bring new ways of doing things if this program has got better and better, so I'm happy to bring this to Steinar and we'll see what Grandmaster says when he comes back.

[no_connection]

if you have like a priority 4 job that needs to be run and you have a priority 0 job that is already going, the exe_manager will PAUSE the priority 0 job, make the highest priority 4 job go through and finish and then UNPAUSE the priority 0 job which will slowly but surely reach the end.

Then that would make any argument to have any higher base priority than normal moot. Setting to high will not make it faster by any stretch of the imagination but will make any other function of FFAStrans or the OS unstable. You are not going to be able to VNC into the machine for example.
And if you don't have iLO or similar hardware remote management you might have to wait until job is done. How do I know that? well I just submitted a job through VNC. FFAStrans gui is unsusable and don't respond. web monitor barely keeps up thanks to being set at above normal base priority.

Anyway, this is to say that I'm not ignoring you and I'm not giving you a "hard no", on the contrary, I agree with emcodem and we will bring this up to Grandmaster, but keep in mind that we developed things this way 'cause there was a reason and it seemed to be working for lots of our user base, but yeah, we're open to new use-cases, after all, it's thanks to our users like you who bring new ways of doing things if this program has got better and better, so I'm happy to bring this to Steinar and we'll see what Grandmaster says when he comes back.

Let me put it like this, the regular high volume user with several server dedicated probably don't submit jobs directly to the work flow. So it won't affect them. However if you where in need to do a smaller job like converting a few files for edit, you probably do that on the computer you are using. So having ffmpeg set to high would be bad for most use cases I can think of.
And since I'm at it I let the encode run with high just to compare with low. Spoiler but it didn't make a difference in encoding time.
Higher process priority than normal don't really make it run faster, and with dedicated servers it's even less of an argument since they don't have any resources to "fight" for to begin with.
I do understand your reasoning with priories and I do *get* why it's the way it is right now. But setting things to high should not be done without very good reason, and most certainly not "because the user said so", or "I want this job done now".

This again is normal behavior and it's expected to work like this 'cause if you're "bold enough" to get into the Workflow Manager yourself and start a job manually, it means that either you want to test something or that you're really really really in a rush, hence the top priority above everything else.

You can argue all you want that I shuold have set up a watch folder, moved the files to there, let it encode, then move them back. Instead of just submitting the 5 files I needed directly to the workflow.

[emcodem]

Such "features" like manual jobs have a different default prio than others should never be something hardcoded but a user decision instead. Also the default value should always be most compatible and least intrusive instead of just matching the expections of a single user. It would in deed be most logic that manual submits follow the workflow prio just like anything else.

... exe_manager will PAUSE the priority 0 job, make the highest priority 4 job go through and finish and then UNPAUSE the priority 0 job which will slowly but surely reach the end.

Exactly, since this feature was introduced, we actually don't keep do this hacky process priority management at all. So if steinar is willing to discuss this (and again rework the stuff), i'll vote for just removing the whole feature. The reason is that it is something that you usually don't see in any software, also no professional transcoder that i know does that - for good reasons. Did you know that ffastrans even tries to set the priority of the SYSTEM process PID 0? If i was an antivirus i'd complain at this point.

If we remove this feature, we can also delete the warning when you set to CPU roof 100% "this will render your machine unresponsible". Maybe if we run lucky we can also remove the whole CPU roof feature along with it.

Anyway, such stuff is decided by steinar only, FranceBB and me can only argue and beg here just like any other user

[FranceBB]

Anyway, such stuff is decided by steinar only, FranceBB and me can only argue and beg here just like any other user

Eheheheh he's called "Grandmaster" Steinar for a reason hahahaha