file processing order
file processing order
Hi,I found this program recently and trying to use it in my workflow.My question is how files are processed? I thought it was like first in first out,but I see that newer files are processed before older are finished which is not good in my case.Is there any option to change this behavior?
Re: file processing order
Hey there!
Assuming you are talking about a single workflow/watchfolder. Are you using the processor "monitor" and set the checkbox "Check growing files" to "once" and also you check the "Forget missing files" option? If yes, new files would typically be processed as soon as writing is finished.
From your question i see that it is also possible that you talk about you want to have only one processing being done at a time which would be set in Options- >Configuration->Host->Max active jobs
cheers,
emcodem
Assuming you are talking about a single workflow/watchfolder. Are you using the processor "monitor" and set the checkbox "Check growing files" to "once" and also you check the "Forget missing files" option? If yes, new files would typically be processed as soon as writing is finished.
From your question i see that it is also possible that you talk about you want to have only one processing being done at a time which would be set in Options- >Configuration->Host->Max active jobs
cheers,
emcodem
emcodem, wrapping since 2009 you got the rhyme?
Re: file processing order
I set up simplest workflow monitor folder->ffmpeg->output folder.For example I have files dated 2 apr,3 apr in that folder when I start workflow and i want these files encode first, also new files are written at the same time ( records from tv broadcast)to the monitor folder.I checked growing files so it does not start encoding before records ends which is great,but once it is finished it starts encoding of new files.
I want new files to be at the end of the queue
I want new files to be at the end of the queue
Re: file processing order
Hi Dmb,
You must know FFastrans starts encode a file when it is complete (as soon as the file is completely arrived in monitor folder (as Encodem wrote to you in the previous post). FFastrans doesn't use creation date or modified date for monitoring(You can use conditional condition AFTER the monitor folder to sort your files date).
About your first post, the rule "first in->first out" is only true if all your files are the same duration. For example, a 10Gb file deposited at 10:00 will be out after a 1Gb file deposited at 10:02
I have to admit I don't understand your final goal...
Cheers.
You must know FFastrans starts encode a file when it is complete (as soon as the file is completely arrived in monitor folder (as Encodem wrote to you in the previous post). FFastrans doesn't use creation date or modified date for monitoring(You can use conditional condition AFTER the monitor folder to sort your files date).
About your first post, the rule "first in->first out" is only true if all your files are the same duration. For example, a 10Gb file deposited at 10:00 will be out after a 1Gb file deposited at 10:02
I have to admit I don't understand your final goal...
Cheers.
Re: file processing order
Hey Dmb and Benjamin 
i think i understood Dmb's request very well and it is a little strange behaviour but i can imagine well that it works as Dmb describes (without reproducing it). @admin might be able to confirm the behaviour.
However, i don't see an easy way to get to your goal. The best thing i can imagine is that you click the "rebuild history" button in your monitor, so it does only process "new" files. And before setting it to "active", you use my webinterface (sticky thread here) and queue all existing files. This way it would behave as you wish. But be aware that a queue of multiple hundreds might cause strange behaviour altough processing might work fine.
cheers!
emcodem
i think i understood Dmb's request very well and it is a little strange behaviour but i can imagine well that it works as Dmb describes (without reproducing it). @admin might be able to confirm the behaviour.
However, i don't see an easy way to get to your goal. The best thing i can imagine is that you click the "rebuild history" button in your monitor, so it does only process "new" files. And before setting it to "active", you use my webinterface (sticky thread here) and queue all existing files. This way it would behave as you wish. But be aware that a queue of multiple hundreds might cause strange behaviour altough processing might work fine.
cheers!
emcodem
emcodem, wrapping since 2009 you got the rhyme?
Re: file processing order
Hi emcodem and dmb,
Ok so it's a cache issue. If you want to keep your old files in your monitor folder dmb, don't forget to set the configuration of your workflow (right-clic on workflow->properties->maintenance->max history cache age = you will set the time of "memory" of FFastrans to avoid process file again).
Hope emcodem and I have well help
Cheers.
Ok so it's a cache issue. If you want to keep your old files in your monitor folder dmb, don't forget to set the configuration of your workflow (right-clic on workflow->properties->maintenance->max history cache age = you will set the time of "memory" of FFastrans to avoid process file again).
Hope emcodem and I have well help
Cheers.
Re: file processing order
Hi dmbf22, thank you for using FFAStrans and welcome to the forum!
FFAStrans has no concept of date-stamps in regards to what file should be processed first. So it will pick whatever file(s) it finds first and start rocessing it IF it's readable. Normally the files will be processed in alphabetical order, bacause that's what windows will give you when enumerating files.
However, I'm not very happy with the logic that cause FFAStrans to start new time consuming jobs before finishing the ongoing ones. This is something I'm addressing in the new major update I'm now working on. It will be a while before it's released so until then, you will have to live with the current behavour.
-steinar
FFAStrans has no concept of date-stamps in regards to what file should be processed first. So it will pick whatever file(s) it finds first and start rocessing it IF it's readable. Normally the files will be processed in alphabetical order, bacause that's what windows will give you when enumerating files.
However, I'm not very happy with the logic that cause FFAStrans to start new time consuming jobs before finishing the ongoing ones. This is something I'm addressing in the new major update I'm now working on. It will be a while before it's released so until then, you will have to live with the current behavour.
-steinar