User Tools

Site Tools


processors:monitor_folder

Folder Monitor

Purpose

This processor is used at the start of a workflow to watch for new files in a Local or UNC Path.


Limits

The settings “Forget missing” and/or “Growing Check continuous” raises the RAM and CPU usage of this processor in case your Watchfolders contain many (e.g. thousands) “already processed” files.
To work around the extra time/CPU load for “Forget missing” in case of huge watchfolders, use a commandline processor with this content: %comspec% /c “if exist ”%s_cache_record%“ del /f /q ”%s_cache_record%“”
Without any of those 2 settings, the Processor was tested for

  • 500.000 Files in a complex subfolder structure
  • it took about 16 minutes to pick up new files

NOTE : All variables can't be used inside this processor, you can only use:

  1. Static Variable
  2. Time Variable
  3. Windows Variable.


Potentially the processor can do more than the tested amount of Files but consider that it will take a long time to pick up new files.


Original Manual



This node is for monitoring local or UNC paths. It will also parse Windows shortcuts and Mac alias to find the correct file. Be aware that the "Recurse" and "Forget missing files" options will cause the monitor, run in a separate process, to use more CPU. This is most notable when monitoring thousands of files.

Folder

Set folder that will be monitored. Supports drag'n drop.

Accept/Deny files

Here you can define what kind of files that will be accepted or denied. Supports wild card (*) and question mark(?) for single character. Multiple file names can be separated by pipe (|). Both "Accept" and "Deny" fields can be defined. Supports variables.

(Accept) Example: «*.mov|boat*.mxf» will seek for any kind of .mov file and any kind of .mxf file starting with «boat»

(Deny) Example: «Thumb.db|*.wav» will skip any .wav files and «Thumb.db» files generated by Windows.

Deny folders

This enables you to skip folders. Supports wild card (*) and question mark (?) for single character. Multiple folders can be separated by pipe (|).

This option requires "Recurse" to be enabled.

Deny attributes

This enables you to skip files marked with any of the RASH attributes.

Create folder

If this is checked, FFAStrans will automatically create, if it does not exist, the folder specified under «Folder».

Recurse

This option seeks into sub-directories of the folder specified.

Localize file

If set, FFAStrans will make a local copy of the accepted files. These are just preliminary files and will be deleted once the job is done.

Check for growing files

The speed of the cycle is defined by the sleep timer setting in the workflow properties.

once: This option will make FFAStrans wait to see if incoming files are still growing. If FFAStrans do not detect any growth since last cycle, the file is considered finished and ready for processing.

continuously: This option will make FFAStrans execute new job every time the file(s) have changed file size. This comes in handy when looking for constantly updated files that need to be processed for each update.

Skip source verification

When enabled FFAStrans will skip trying to open the file for reading in order to verify if it's ready or not.

Forget missing files

With this option enabled, FFAStrans will synchronize the files history to the actual files. This enables you to remove an already processed file, wait a sleep cycle, and add the same file back again and it will be picked up.

Limit file size

Here you specify the maximum file size accepted by the monitor.

Rebuild history

This button make FFAStans register files already present in the monitored folder. Use this if you want to skip existing files prior to starting the workflow.

Clear history

This does the opposite of "Rebuild cache". Any files registered by the monitor will be unregistered, and thus FFAStrans will start processing the files once again. This is equivalent as "starting from scratch".

2019/12/09 16:19
processors/monitor_folder.txt · Last modified: 2021/10/31 16:09 by benjamin

Donate Powered by PHP Valid HTML5 Valid CSS Driven by DokuWiki