set order in which folders are processed

[FranceBB]

Guys, FFAStrans auto pausing (in the middle of processing) is turned off by defalut so for "regular" users it's not enabled.

I forgot it was only enabled in a specific branch. Epic fail on my part.

[Lugi]

@admin/steinar: The easiest way to implement may be if there is a global limit option for jobs and an option that higher priority workflows are processed first until nothing is to do anymore, and only then the lower priority workflows are processed.

But then it would be nice if there is a sperate processor priority option.

[emcodem]

The easiest way to implement may be if there is a global limit option for jobs and an option that higher priority workflows are processed first until nothing is to do anymore, and only then the lower priority workflows are processed.

This sounds very much exactly how it is right now?

[admin]

Yes, looking at this again now and it's true that this is actually how it works today. So the "old" behavior is basically the same with the difference being that higher priority jobs will start even if you have used all your job slots. Then the jobs slots will be filled with the higher pri jobs before doing the rest. That means if you have available 3 slots, 2 high pri jobs and 2 low pri, then FFAStrans will start the two high pri jobs and one of the low pri ones. So a different approach would be to NOT start any NEW lower pri jobs as long as there are higher pri job(s) running, even if you have free job slots and resources.

-steinar

[Lugi]

I had made two workflows with two different priorities and with a job limit of 2 in the settings. With this, FFAStrans then started four jobs at the same time, two of each priority. That, in my opinion, is stupid behavior of the software. I would like if the software with those settings only starts two jobs of the higher priority. Only if there is nothing to do in that priority, the software should start jobs with the lower priority.

[Lugi]

Now I have an example where jobs get stuck. My hard drive went full. So all jobs stop (including the jobs that would delete files -_-). After I made space free, over halve of the jobs (that would delete files) keep being stuck. They don't resume.

[admin]

Making sure you have enough HD space is a must for any system. FFAStrans use the file system as is and depend on that it's working as expected and have enough free space. Now that you managed to get into the case of full HD, there will be a bit of cleaning up in the FFAStrans database, which with the current design use the file system heavily. The most important thing to do is to take a look a the processors\db\cache\tickets and processors\db\cache\monitor folders. If you see any .json files in there with 0 bytes (caused by your system having no free space), then just delete them. If you have stuck jobs it might just be db entries not being cleaned properly (again because of the free space issue). If that is the case delete the corresponding json files in processors\db\cache\monitor.

About FFAStrans priority and inner workings: FFAStrans assign the number of set slots (per host) to each of the priority classes. So both lower and higher have the same free slots. However, lower pri jobs will only start if there are free job slots for that and any higher classes. So when you have set job slots to two (2) and you have filled up with higher pri jobs, then lower pri won't start until the higher is finished. BUT, it will not prevent lower pri jobs to start before higher pri jobs enter the queue. So it looks like this is what has happened in your case. Lower pri jobs enterted the queue first, but because higher pri jobs entered after that they also started. So you have then 4 jobs running even if you set only two slots. Now, the two higher pri jobs will be given higher resourse priority and start any subsequent nodes before nodes in the low pri jobs.

Hope this helps you understand how FFAStrans priority currently works.