Page 1 of 1

accepts only files that are newer than a given date?

Posted: Thu May 23, 2024 9:48 am
by bob
Hi!

Is it possible to configure a watchfolder to accepts only files that are newer than a given date?


I have a bunch of files in a directory, but I would like the encoding to start from May 22 and include all new files that arrive in the folder.



many thanks for your help

Re: accepts only files that are newer than a given date?

Posted: Thu May 23, 2024 10:41 am
by emcodem
Hi @bob,

currently this would require a more complex setup: a scheduled job on webinterface and a workflow with the plugin processor "files find".
In this case the workflow would look like:

Code: Select all

                                                  Yes? foreach - restofworkflow
                                                /
files find  - conditional (are there new files?)
                                                \
                                                   No? - Dispel the job(Delete from job monitor)
Basically you would see a very short running job every minute this way because the "check if there are new files" is now part of the job instead of being done by some hidden service (def_runner). But if no files were found, the job would delete itself from history.
Let me know if you want to go down this more complex route so i know if it pays off to work out some example workflow for you if you need one.

Re: accepts only files that are newer than a given date?

Posted: Fri May 24, 2024 11:15 am
by bob
I would be very grateful, I have already installed the webinterface and plugin processor

Re: accepts only files that are newer than a given date?

Posted: Mon May 27, 2024 6:45 am
by emcodem
it appears that i could not find some example that i did for this in the past so it will take a little longer than i thought but i'll send something soon.

Re: accepts only files that are newer than a given date?

Posted: Fri May 31, 2024 5:29 am
by ThomasM
Hmmm... Beside the fact, that a Date/Time-Feature in the Monitor-Nedes would be great, I would suggest a PowerShell-Script after the Monitor-Node to compare the dates with the Get-Date CMDLET. A brief explanation that was helpful for me is here:

https://www.sharepointdiary.com/2020/08 ... shell.html and of course all the entries on StackOverflow-Forum.

The output of the PS-Script is then put into a UserVariable that can be used for a ConditionalNode to either process the file or to Dispel the job.

Maybe this is a starting-point?

cheers,
Thomas

Re: accepts only files that are newer than a given date?

Posted: Mon Jun 03, 2024 3:29 pm
by emcodem
The way that @ThomasM mentions is possible, you would need to delete the cache record if the file is not old enough and dispel the job so it can be picked up at next run again. The difference to my design is that you would see one job per too young file instead of one job for all files, which means if you have many files waiting to become old enough, you would also see many job every minute. In my design you see only one job per minute, independent of the number of files that are waiting.

I just updated the files find processor for you @bob and added an option called "Only new Files". This will cause the files find proc to remember which files it already picked up and output the same file only once (unless it has been removed from the watchfolder for one run, in that case the filename will be deleted from the watch database, similar to the normal monitor)

Please update the files find processor to the newest version!
emcodem_files_find_by_age.json
(8.72 KiB) Downloaded 71 times
In this workflow, the upper path should be scheduled by webinterface to run every minute or similar. If a new file was found, the upper path will create a new job for each found file starting at the "lower" path.
The lower path (a single populate processor) is where your actual file processing starts.

You should leave the populate processor as first processor of your workflow, otherwise you would need to change the ""start_proc" id in the http comm processor on the upper path.

In a more advanced example of this we can use webinterface type variables to let you define the watchfolder in the scheduled job configuration on webui instead of inside the ffastrans workflow. This way you can have multiple watchfolders in form of multiple scheudled jobs instead of having a copy of the workflow.
Let me know if you need an exapmle for such a configuration please.