local caching - long time to finish a job

Here you can submit bugreports
Post Reply
taner
Posts: 204
Joined: Sun Jun 19, 2016 6:36 pm

local caching - long time to finish a job

Post by taner »

Hi Steinar,

I've tested local caching because it drastically improves transcoding-speed (depending on the hardware).
I dropped a dozen files into a watchfolder monitored by all my 5 transcoding-machines.
So far so good.
But sometimes I had to wait a long time until a job was finished (compared to previous setup with global shared media cache).
Example: a machine picks up a file, then in the status-monitor is shown "waiting for (next) processour ressources", then transcoding starts, then again "wait for free processour ressources"...and so on.
Actually quite normal waiting for processor ressources but it takes sometimes a couple of minutes until the machine moves to the next step.
This does not change even if I reduce maximum jobs to a single job for each machine.
The machine has only this one job to do at that moment.
It does not only happen with specific files or machines.
Is this a normal behaviour?

Best
Taner
admin
Site Admin
Posts: 1687
Joined: Sat Feb 08, 2014 10:39 pm

Re: local caching - long time to finish a job

Post by admin »

Hi Taner,

No, this is not normal beavour. I use the same configuration on one of my installations and it's quite snappy moving from processor node to processor node. That is if the CPU is not too loaded by other processes. I assume you got that under control?
Can you please send me the ffastrans.ini file so I can see if there are any anomalies there?

-steinar
taner
Posts: 204
Joined: Sun Jun 19, 2016 6:36 pm

Re: local caching - long time to finish a job

Post by taner »

Hi Steinar,

in my first attempt i changed the caching method "on-the-fly".
All workflows were running.
This was not a good idea.
Stopping all workflows, quitting FFAStrans (all instances were set to "run as application"), starting FFAStrans and the workflows again did not help.
After rebooting the machines all works now as expected.
Fast moving from node to node.
My machines are so happy now that they transcode twice as fast.
Great feature!

Thanks

Taner

Edit: when i submit files they went into the queue of the machine where i submitted them. I assume that can't be changed? Meaning distributing the submitted files to all transcoding-machines.
admin
Site Admin
Posts: 1687
Joined: Sat Feb 08, 2014 10:39 pm

Re: local caching - long time to finish a job

Post by admin »

I know there are some issues when submitting from a PC when running in passive mode. But if you submit on one of the active hosts then the submitted file will behave as if it was picked up by that host. So, if you have configured the host to use local cache then the job will be taken by the local host only. If you have set the host to use shared global cache then the job will be picked up by any hosts with the same configuration.

-steinar
Post Reply