FFAStrans v1.4.0.7 - folder monitor sometimes stops seeing files

Here you can submit bugreports
User avatar
Silicon
Posts: 108
Joined: Fri Sep 04, 2020 6:34 am

Re: FFAStrans v1.4.0.7 - folder monitor sometimes stops seeing files

Post by Silicon »

Hi admin
I have finally identified the root cause ... the problem is not the actual length of the filename, but the length of the unc path of the json file created by the monitor node, exceeding 259 chars.
Example:
  • file dropped to watchfolder: 12345678901234567890123456789012345678901234567890123456.mxf = 60 chars incl. ext
  • UNC path to JSON created by monitor node: \\myffastransmng.subdomain.company.com\FFAStrans\Processors\db\cache\wfs\20260327-1939-1526-6bd4-4187805bccfe\mons\20260327-1939-4765-3959-8977285aa843\12345678901234567890123456789012345678901234567890123456.mxf_D9179654900785B357721011E405C5BB01DAE917.json

This JSON is left intact in mons folder i.e. file dropped to wachfolder is not processed
BR,
Silicon
--------
FFAStrans 1.4.1 WebInterface 1.4.0.187
Manager: VM: 4x VCPU, 12GB RAM, Windows Server 2022
Workers: 2x Dell PE R7615 (AMD AMD EPYC 9354P 32-Core @3.25GHz, 64GB RAM, nVidia L4, Windows Server 2022)
admin
Site Admin
Posts: 1704
Joined: Sat Feb 08, 2014 10:39 pm

Re: FFAStrans v1.4.0.7 - folder monitor sometimes stops seeing files

Post by admin »

That makes more sense! :-) Do you have an option to shorten the FFAStrans install path? In the mean time I will look at this in order to properly support long paths also on the install path.

-steinar
User avatar
Silicon
Posts: 108
Joined: Fri Sep 04, 2020 6:34 am

Re: FFAStrans v1.4.0.7 - folder monitor sometimes stops seeing files

Post by Silicon »

Yes I have used hostname instead of FQDN which saved 22 chars.
If I may I would recommend trimm of the source filename when constructing the mons json filename
BR,
Silicon
--------
FFAStrans 1.4.1 WebInterface 1.4.0.187
Manager: VM: 4x VCPU, 12GB RAM, Windows Server 2022
Workers: 2x Dell PE R7615 (AMD AMD EPYC 9354P 32-Core @3.25GHz, 64GB RAM, nVidia L4, Windows Server 2022)
admin
Site Admin
Posts: 1704
Joined: Sat Feb 08, 2014 10:39 pm

Re: FFAStrans v1.4.0.7 - folder monitor sometimes stops seeing files

Post by admin »

Sure, that's an option but not safe as the total path can still be too long. Also, it would mean converting all cache records files...which is something we should really try to avoid. Anyway, this is not a too normal case so it's great that you can "fix" it temporarily.

-steinar
emcodem
Posts: 1918
Joined: Wed Sep 19, 2018 8:11 am

Re: FFAStrans v1.4.0.7 - folder monitor sometimes stops seeing files

Post by emcodem »

Just FYI @Silicon, i explained what the situation and what we currently did to steinar. He is now exactly where i was: cannot reproduce.
Let's see if he comes up with some ideas to further debug this.
emcodem, wrapping since 2009 you got the rhyme?
User avatar
Silicon
Posts: 108
Joined: Fri Sep 04, 2020 6:34 am

Re: FFAStrans v1.4.0.7 - folder monitor sometimes stops seeing files

Post by Silicon »

Hi emcodem
What do you mean by “cannot reproduce”?
Does it mean, that you are able to process mons JSONs with UNC longer than 260 chars in your enviroment?
Does autoit support long paths?
BR,
Silicon
--------
FFAStrans 1.4.1 WebInterface 1.4.0.187
Manager: VM: 4x VCPU, 12GB RAM, Windows Server 2022
Workers: 2x Dell PE R7615 (AMD AMD EPYC 9354P 32-Core @3.25GHz, 64GB RAM, nVidia L4, Windows Server 2022)
emcodem
Posts: 1918
Joined: Wed Sep 19, 2018 8:11 am

Re: FFAStrans v1.4.0.7 - folder monitor sometimes stops seeing files

Post by emcodem »

Oh Sorry there was an overlap of things, after i talked with steinar, you posted the infor that its about >260 char paths and "long" install dir and i didnt read this update before writing above.

So basically support for "long paths" under windows is not really something the application decides, afaik all applications that use standard file apis e.g. "fopen" underly the same restrictions. So it's not really an autoit thing to support or not support long filepaths for the standard File apis but a microsoft thing.

However, microsoft gave us the option to enable long path support in any exe that uses the stndard file apis. This is done by just embed a manifest to make it work (ask the LLM for "how to embed a manifest into exe for longpath support"), the following exes that match your versions seem to do the trick for me, tested with a ffastrans install path like:c:\ffastransffastransffastransffastransffastransffastransffastransffastransffastransffastransffastransffastransffastransffastrans

https://drive.google.com/file/d/1rk9j0g ... sp=sharing
emcodem, wrapping since 2009 you got the rhyme?
User avatar
Silicon
Posts: 108
Joined: Fri Sep 04, 2020 6:34 am

Re: FFAStrans v1.4.0.7 - folder monitor sometimes stops seeing files

Post by Silicon »

Hi emcodem
Thanks for your explanation.
Let me inform you that since the configuration change (hostname instead of fqdn) all monitors are picking up the files normally.
BR,
Silicon
--------
FFAStrans 1.4.1 WebInterface 1.4.0.187
Manager: VM: 4x VCPU, 12GB RAM, Windows Server 2022
Workers: 2x Dell PE R7615 (AMD AMD EPYC 9354P 32-Core @3.25GHz, 64GB RAM, nVidia L4, Windows Server 2022)
emcodem
Posts: 1918
Joined: Wed Sep 19, 2018 8:11 am

Re: FFAStrans v1.4.0.7 - folder monitor sometimes stops seeing files

Post by emcodem »

Damn i was hoping we get a second confirmation that the manifest stuff works so steinar adds this to his build and we solve the topic once and for all :D
emcodem, wrapping since 2009 you got the rhyme?
Post Reply