Checking growing files continuously missing some small files

[jonathonz]

Hi,

I turned on "check growing files continuously" on my UNC watch folder after observing a file get moved prematurely that was being FTP'd into the watch folder that was still growing.

However after I did this I noticed that the small 10KB .xml files sporadically were not getting picked up anymore.

Any tips to troubleshoot this behaviour?

Running with two nodes and storage on Isilon via UNC.

Thanks!

[emcodem]

Hey,

one question: before you checked "check continuous", did you have checked "check once" or did you not have the check option enabled at all?
From my understanding, "check once" should be the default that everyone should check but "check continuous" should only be checked in extremely, veeery specialized cases where you expect the source file to be changed multiple times instead of just being transferred to the watchfolder...

Cheers,
emcodem

[jonathonz]

Hi,

I had check once turned on, and only tried check continuous after seeing the file move prematurely.

I moved it back to check once as it the premature move only happened once, while found check continuous wasn't reliable.

[emcodem]

Hm, might be that the file was processed prematurely because the "check time" was too small.
@steinar: is the "check" time directly connected to the "sleep timer" of the workflow settings?

cheers,
emcodem

[admin]

Hi all,

Yes emcodem, the check time is connected to the sleep timer of the workflow. So setting too low with growing files enabled might cause files to be picked up prematurely. Mind you, determening when a file has grown "enough" will never be 100% fool proof. There will allways be cases where the file was not finished after all. Fine-tuning is an always ongoing process and the "continuously"-option is quite new and so it might not be fine tuned enough yet.

-steinar