Copy files from storage to server with checks?

[stuvmark]

Hi, I'm wondering if it possible to get FFAStrans to help me do a couple of checks when copying files between to servers. The recorder records mp4-files and creates md5 checksum files in it's folder after finished recording a file. What I want is a way to check that the files is not corrupt, and have a duration between lets say 59mins58secs and 1hour02secs.
If the file is "valid" (not corrupt) and between the durations set, deliver to one folder. If it fails, copy it to a CheckFailed-folder.

Today I have a folder that picks up a file after it has checked that the file has been untouched for a while.

Sorry if this is easy for you, but I'm not that advanced in scripting/programming.

[emcodem]

Hi stuvmark,

what you mean by chck if file is "valid", you just want to compare the md5 or check if there are error messages when decoding it with ffmpeg or similar?
If MD5 check only, can you give an example for file naming (mp4 and the md5 file)?

For the duration check, this one should be more simple. If you start your workflow from watchfolder (listen for *.mp4), you insert a conditional processor checking 2 things:
%f_duration% > 3540
AND
%f_duration% > 3602

The numbers i wrote are 00:59:58 and 01:00:02 written as seconds.
After this conditional processor, if the condition is OK, the workflow would go on with next processor that has a green input connector. If it fails, it would just write an error message to status and not continue processing. If you want to do something on error, you would insert some processor and change the input pin to red color.

Make sure you provide the requested information about the checks you want to do so eventually i can come up with some example workflow.

[ThomasM]

Hey all,

should the second duration be smaller?

For the duration check, this one should be more simple. If you start your workflow from watchfolder (listen for *.mp4), you insert a conditional processor checking 2 things:
%f_duration% > 3540
AND
%f_duration% > 3602

Code: Select all

%f_duration% > 3540
AND
%f_duration% < 3602

cheers,
Thomas

[emcodem]

Ups, yeah thats correct thanks for the correctio thomas (you passed my test )

[stuvmark]

Hmm, the MD5-file can be copied using a designated workflow that only picks up those files, so lets forget about the MD5. By "vaild" I mean not corrupt. We have encountered trouble where the recorder has lost control over the video file and that ends up with a corrupt file unreadable by most video players. So what I'm thinking is:
1. Folder pickup - waits for the file to be ready, picks up .mp4-files only.
2. Check that the file is not corrupt. If corrupt, deliver to "corrupted"-folder
3. Check duration. If duration is outside of scope, deliver to "deviation"-folder
4. Deliver file to "normal" destination. If it hasn't failed on the two tests.


Number 2 and 3 can be combined into a failed-folder, doesn't really matter.

[emcodem]

OK thanks for the clarification.

So please check this out and let us know any questions.

emcodem_file_valid_check.json

(4.69 KiB) Downloaded 316 times

The duration check here i do for >0 and <3600 seconds check if we have at least 1sec but not more than 1 hour duration, you can change to any seconds value you like.
The second check, commandline ffmpeg like this:

Code: Select all

"%s_ffmpeg%" -i "%s_source%" -xerror -f null -

What this does is to "decode" the whole file and exit with error (xerror) if there was a decoding error. I tested by submitting some media file -> ok and some zip file -> not ok.

[stuvmark]

This seems to work perfectly. I have tested renaming some non-video file to .mp4 - it "failed". I also tried valid mp4-files that was too short and too long, and they also "failed". It seems to process regular files as expected. Just have to dismantle what you have done to understand why stuff happens, as one of the processors deliver to two others. But that's just for my curiosity. Thanks a lot, I know some SW companies that are not free that are way slower and doesn't provide this level of service when you ask questions that's not in their FAQ.

[emcodem]

Thanks for the flowers:D

...to understand why stuff happens, as one of the processors deliver to two others. But that's just for my curiosity.

At the bottom of this page there are some words about error handling:
https://ffastrans.com/wiki/doku.php?id= ... &s[]=error

[stuvmark]

Ahhh, didn't even know about that - it explains everything!