Webinterface

[Stef]

I did notice something in the Job Submitter, files are always sorted by creation date, descending. Clicking on a header sorts by that type, but changing the directory always resets sorting back to creation date.

Thanks a lot for giving it a shot Stef! What you say is true, unchangeable currently. I believe i did save the sort order in some older version but i reverted because it caused troubles and felt kind of weird for larger shares. However, of course you have a valid point here, i might want to re-enable saving the sort order but maybe only for the current session or for the current browse location? What you think?
Also, if you re-order, what do you usually use, "name"?

Hey @emcodem

I think saving it for the current session would be a good idea, or a changeable default setting(?). Per folder seems like it would be overkill to save that for every single browsed folder. Usual sorting (in my case atleast) would by name, indeed.

Been looking a bit further, and I've noticed some other things - just giving feedback, not complaining

The job overview also sorts kind of oddly. Active jobs are at the bottom, and the queue seems to get disorganised over time (older and more recent submitted jobs get jumbled). In previous versions, it seems like the view was sorted by state first -so they would be at the top -, and then start time, ascending - oldest queued jobs are right below the active ones, since they would start first after the current ones are finished.
-- though as I'm writing, that doesn't seem to happen right now? EDIT: Again, screenshot attached below.

Screenshot Queue.png (24.56 KiB) Viewed 943 times

These are pretty much nitpicking though, this kind of thing is up to preference, I know Functionality stays the same.

Sometimes there's an error getting queued jobs ("Error getting queued jobs. If FFAStrans installation moved, please correct Path in Admin settings.)". I noticed however that the API (ip:3003/tickets) returns an error at that point, ENEONT no such file or directory. It complains about a .json file not existing in the FFAStrans folder. Im not sure if that is something unnique on my end though, seems to happen only when serveral jobs are processing. (I did change my network setup to improve responsiveness, as suggested by FranceBB and yourself. WebUI and FFAStrans processing are completely split, processing cache is separated too).

Example output:

Code: Select all

{"description":{"errno":-4058,"code":"ENOENT","syscall":"stat","path":"\\\\192.168.10.90\\FFAStrans\\Processors\\db\\cache\\tickets\\running\\0~60e8b42c-aed0-4719-a708-df3992d0213e~981001100~1-0-0~20240329-1600-3110-86c4-c0446b694656~PRECUT-01~350478.json"}}

Variable editing in the UI works nicely in the fixed version. Being able to sort form fields and customise size is great! Makes for some neat forms.

Screenshot Forms.png (17.6 KiB) Viewed 945 times

[Stef]

Continuing exploring, I noticed that there can be missing jobs in the history view. I think it's caused by some sort of parsing issue, but I don't know what exactly causes it. Below is some output in the database.log file, but I'm not sure if it's relevant. At the time the jobs in the history stop appearing, the webinterface log starts getting these warnings too.

Missing jobs DO start showing up if I clear some job log folders in the FFAStrans db/cache/jobs folder, but I was silly and actually deleted them, instead of moving those somewhere else, can't check those anymore now... I'll report back when I get more info.

Code: Select all

2024-10-15 03:37:35: error: [database]: uncaughtException: Cannot read properties of undefined (reading 'undefined')
TypeError: Cannot read properties of undefined (reading 'undefined')
    at Request._callback (C:\FFASTR~2\node_components\cron_tasks\jobfetcher.js:194:32)
    at self.callback (C:\FFASTR~2\node_modules\request\request.js:185:22)
    at Request.emit (node:events:513:28)
    at Request.<anonymous> (C:\FFASTR~2\node_modules\request\request.js:1154:10)
    at Request.emit (node:events:513:28)
    at IncomingMessage.<anonymous> (C:\FFASTR~2\node_modules\request\request.js:1076:12)
    at Object.onceWrapper (node:events:627:28)
    at IncomingMessage.emit (node:events:525:35)
    at endReadableNT (node:internal/streams/readable:1359:12)
    at process.processTicksAndRejections (node:internal/process/task_queues:82:21)

Code: Select all

2024-10-14 18:01:10: warn: [webint]: Over 200ms response Time (259,947 ms), GET /api/json/v2/jobs/?start=0&count=100
2024-10-14 18:01:10: warn: [webint]: Over 200ms response Time (457,302 ms), GET /api/json/v2/jobs

[FranceBB]

Sometimes there's an error getting queued jobs ("Error getting queued jobs. If FFAStrans installation moved, please correct Path in Admin settings.)".

Yep, that's something I noticed as well and reported privately to emcodem a few days ago, but I'm really glad that you could reproduce it as well!

Screenshot from 2024-10-15 12-57-13.png (454.97 KiB) Viewed 876 times

emcodem actually said that he's still working towards the migration from API to reading from file storage and that the issue we're seeing there is given by the APIs.
The very good news, however, is that we won't get that error for much longer.
To quote him:

The new approach cannot timeout, synchronisation happens in the background so timeouts can only occur on server side but not on client side.
I am seeking for the last fragments reading stuff from api right now

In other words, using the new approach, we won't ever see anything timing out as it will read from the storage, so unless getting files from the actual storage times out (unlikely and in that case you would have a much bigger problem than the web interface), it will never display an error.
As soon as the new file-based version is released, we won't face that error.

[emcodem]

@Stef

thanks again for all the investigations, you seem to do serious business with webint

Here is some prerelease for you, it should store the job submitter file sorting globally even if i noticed that windows explorer does it per folder.
However, the most urgent thing is the "sometimes miss jobs" stuff, i can't allow this to happen. There are already some mitigation strategies involved in this prerelease but unless i know the exact reason for your issues, i am not 100% certain about the fix.

- EDIT: something went terribly wrong with this upload, will re-up tonight. -

Maybe if you experience the issue again, you can zip and send me webint.log file?

Also you have a point about active jobs sorting, i shall re-visit that part. I believe currently it is just sorting by state and the rest is more or less accident.

And that's a nice job submitter form you got there, exactly how it's intended to be...

[emcodem]

@Stef

sorry if the mistakes with the "prerelease" caused confusion and efforts.
This should work:
https://github.com/emcodem/ffastrans_we ... 4.0.67.zip

[Stef]

Hi @emcodem

Missing jobs in the history view happened again. I moved job folders out of the db/cache/jobs folder to see when the monitor recovered, there are few possible culprits. I have collected the logs aswell.

That was still on version 1.4.0.40 though, haven't got time yet to upgrade to the prerelease version yet... (probably this week).

Sent you a .7z archive in DM.

[emcodem]

@Stef

i found the reason for your problem, it was when a job has more than 100 branches, the history would not update anymore, especially if it was a manually submitted job.
I made a prerelease for you but this change is so important that i play with the thought of releasing this one.

https://github.com/emcodem/ffastrans_we ... 4.0.85.zip

[Stef]

Thanks for the update, the job history seems to update okay again.

Yea, I've got a workflow that contains a Files Find node that can explode in alot of branches. The folders contains subfolders with each a single clip (RED Camera), so to prevent having to submit possibly hundreds of single files by hand, I use that node to search for them automatically. But that can result in something like 300-400 branches... Every branch/file just ends in resubmitting the individual file via the HTTP API, so all those branches do end quickly.

[DCCentR]

Hi guys, thanks for wonderful this project!

Noticed the following problem - current tasks are not displayed on the web page:

1.jpg (598.33 KiB) Viewed 107 times

But in status monitor all is ok:

2.jpg (273.54 KiB) Viewed 107 times

It wasn't always like this, originally it was okay. But after a while, it started.
In log i see this errors:

ERR.jpg (254.42 KiB) Viewed 107 times

It's fresh install of FFAStrans v1.4.0.7 & Webinterface v1.4.0.85. FF Install dir on SSD and webserver is on the same machine in same folder.

My logs:

logs.zip

(257.79 KiB) Downloaded 4 times

Can you tell what is the problem here?

UPD: oh and it seems finished jobs also stops updating on web

[emcodem]

Hi @DCCentR

sorry for letting you run into this problem in first place and thanks for the report, perfect that you deliver the webint log along.
So my guess would be that you only see this behaviour when there are jobs running, taking away CPU and especially Bandwith to your NAS09 share.
It is of course important for me to support such problems but it is kind of hard to test, can you please confirm try raising the "API timeout" in webui->admin->Network settings from 7000 to e.g 120000? Meanwhile i'll reactivate some old Raspberry Pi and configure it as WLAN NAS so i have something slow for testing this kind of Situation.

What exactly happens:
So webint reads lots of files from the job history folder (history jobs can currently block active jobs update). When there is network activity, the Latency to the share raises, e.g. if you have 2000 jobs in the ffastrans history and usually each one takes 0.1ms, the response usually takes 200ms, so far so godd but as soon as you have network activity on the same netwok cable, the time for each file raises and we soon have 10ms per file or more, leading to a very long update time.
Of course i have already multiple strategies to try to cache stuff and avoid this but looks like some of it is not working as expected.