How to make a node to wait until the last file in a folder is processed ?.

[rdubo]

Hello guys.

Rodrigo again. I'm very excited putting some simple enhancements to our file based ingest flows here. Now, there's a stone inside my shoe about some other WF that i'm trying to build. How can I put a node to wait until the last file in a folder is processed ?. For example, I need to convert to XDCAM-HD the whole content of a folder with some GoPro files inside and I want to backup the original folder with the MP4 material inside into another location and for that, I need to wait until the last file on the folder is processed before the move. I know you already have an idea of what I need here, but I tried conditionals and hold nodes and I just can't make it happen.

As always any suggestions are well received and appreciated.

Thank you again guys and have a nice time.

Big hugs,

rdubo.

[admin]

Hi Rodrigo,

Is it predictable what the last file is? Is there a number count or maybe a special file name?

Normally, I would just move the files once they are processed but I don't know the reason why you need to wait for the last file. Maybe if you could enlighten upon that, helping would be easier.

-steinar

[rdubo]

Hey, thank you 4 yr quick answer. Well, the truth is that I think there's not much ways to predict how the last file on the folder is named because our customers will bring the SD cards with a variable amount of MP4 files inside and the behaviour of the application is that once the 1rst file of the folder is completed for a node, the next node will act instantly. So, if i want to move the entire source folder I need to wait until the last file on that folder is processed, otherwise the source folder will be moved after the 1rst file is out of the previous node. I tried with hold and conditionals with no success. Is there a way to count, sort and list the files inside a folder, obtain the last name and then apply a hold for a that file ?. I don't know how to do that yet of if it's possible, that's what I'm trying.

I hope this can explain better what I'm into.

Thank you always.

rdubo.

[admin]

I understand what you try to accomplish. The problem still remains; we have no way of knowing what the last file is. If you knew that, it would be easier but I understand that it's impossible. But is moving each file on completion not an option? That would be much easier.

Another approach is to count the files in both the monitored folder and the output folder by utilizing the $exists() function which returns the file count of the file(s) specified. F.ex. if you monitor the folder "H:\CONTENTS\CLIPS" with 7 .mp4 files then the function $exists("H:\CONTENTS\CLIPS\*.mp4") would return 7. So you can compare the counts on both destination at the end of the workflow and IF they match, then it's probably safe to say that the last file is finished processing. Use the "Conditional"-node to compare:

If $exists("H:\CONTENTS\CLIPS\*.mp4") = $exists("D:\Output\*.mxf") Then -> move the folder with at "Command executor"-node

-steinar

[MajorBubble]

Isn't there a way via DOS command to check the growing size of the folder? Then move on when growing stopped?

[admin]

Of course you can write a custom script that checks if a folder is growing and that might be the way to solve this issue if it's to complex to be solved by using FFAStrans. But as I have understood the problem, it is not so complex that you need to use a custom script. It's still not clear if each file can be moved upon completion or not, which would be the simplest solution.

-steinar

[veks]

Hi,
I have a similar but different question.
If files are being transcoded in parallel, how to run the script when all files are finished and not for each transcoded file (which happens now)?
As script needs to run only when all files are transcoded and delivered to the folder.

Thanks!

[emcodem]

Hey veks,
up in front please open your own new topic when you have a different question:-)

Your question sounds like the solution is to use a "hold" processor and check the "sychronize" option. On the input of this hold node, you connect all branches that you want to wait to end before you go on after the "hold" node.

[veks]

Hey veks,
up in front please open your own new topic when you have a different question:-)

Your question sounds like the solution is to use a "hold" processor and check the "sychronize" option. On the input of this hold node, you connect all branches that you want to wait to end before you go on after the "hold" node.

Thanks for your reply.

Problem with hold (it seems) is that when holding finishes it still runs next nodes number of times how many transcode nodes are there.
Check this: http://prntscr.com/qlbu6o

[emcodem]

That is a really good question!
There are many ways to overcome this but typically users don't mind if the nodes after join are executed multiple times. E.g. why would it be a problem for you that the text file is overwritten 5 times?

One example solution:
You take the first branch and insert a populate processor after delivery, create and set the user variable "main_branch" to "1". After the "hold" node that synchronizes all jobs, insert a "conditional" and check if the variable "main_branch" is "1". This will cause only 1 branch to go on and 5 branches to "fail".

If you really want to dive into this with me, let me know your questions please.