Page 1 of 1

Set priority with node

Posted: Tue Aug 04, 2020 7:43 pm
by ddehoff
I have a workflow that encodes multiple resolutions (360,720,1080,4k) from one input file. Whenever a file is uploaded, I want to get the 360 version encoded ASAP so that the user has something to view. Unfortunately my queue can get filled up with videos which are rendering to 4k and no new videos can get encoded to 360. The only way around it seems to be to duplicate my entire workflow for each resolution and give the 360 workflow the highest priority. This is less than ideal given the duplication and the extra work when settings need to be changed for all workflows. It also means that if I need to stop a particular file from rendering, I have to stop it 4 times.

If I could use the Populate Variables node to set a system variable called %priority%, it would allow me to have a branching workflow with different priorities for each resolution. Alternately a Set Priority node would work as well, as would having a priority setting in the encoder node.

Thank you.

Re: Set priority with node

Posted: Wed Aug 05, 2020 3:48 pm
by admin
Hi ddehoff,

Your request is noted. However, it should be fairly easy to fix your issue with a small redesign of your current workflow. If you populate a %s_user_variable% with %s_source% and connect your first 360 node. Then connect another populate node after the first encoding and set %s_source% to %s_user_variable%. Then you connect the others (720, 1080, 4K) to this one. This will ensure the 360 is always encoded first.

Hope this helps.

-steinar

Re: Set priority with node

Posted: Wed Aug 05, 2020 5:03 pm
by ddehoff
Thank you very much for the reply. For videoA, I do have it setup to encode 360 first. The issue is while the 4K version of videoA is encoding and someone submits videoB. I want the 360 version of videoB to take precedence over the 4K encode of videoA. Also, if "Max active jobs" is set to 2 and videoA and videoB are both still working on their 4K encodes and someone submits videoC, I want the 360 encode of videoC to go ahead before videoA and videoB have finished.

Re: Set priority with node

Posted: Wed Aug 19, 2020 8:57 pm
by admin
Right, ok so I see what you mean. So for the upcoming version 1.1.0 much work has been done to improve how priority is dealt with. With the next version you will be able to create a workflow with high priority for the 360 and another for the rest. FFAStrans will then always prioritize the 360 workflow over the other even though you used all you "Max active jobs" slots on low priority jobs. Lower priority jobs will then be given much less resources and will in effect pause or move slowly until high priority jobs are finished.

It's not exactly what you ask for but I should provide much better resource and priority control than the previous versions. Then later we can take a look at even more control at node level.

-steinar

Re: Set priority with node

Posted: Fri Sep 04, 2020 3:21 pm
by ddehoff
This sounds really great as a first step, thank you! However, I'm still excited about branch level control so I can bring my multiple workflow jobs back into one branching workflow.