Monitors:Folder deny files by using variables

[michael85]

Hello everyone,

I'am planing to use FFAStrans as a farm and have to transcode a lot of daily new generated ingests to mp4/h264.
The footage comes automatically to a specific folder which is monitored in my workflow. In this folder there are some files which should not be transcoded to h264. The only value I see to separate these files is their duration. Because these files have always the same duration of 00:59:30:00.
So I tried to use the variable "%s_duration_tc%" but I don't know how to build the duration mentioned above into this variable.

I also want to deny files for transcoding to h264 which were generated before a specific date. The time variables don't seem to be able to do this.
can someone please help me to get my workflow to run as desired?

Thanks in advance

[ThomasM]

Hi,

have yout tried a ConditionalNode?

IF "%s_duration_tc%" IS <> TO "00:59:30:00" THEN

Set "Dispel job instance on false/failed outcome" so the Job is cancelled if TC-Duration is 00:59:30:00
Set the Next node to "Execute on Error" (Right-Click the green InputNodeConnector to set it to red) <- sorry, was wrong...

For further information check the Wiki-page for Conditional-Node: https://www.ffastrans.com/wiki/doku.php ... nditionnal

just my two cent...

regards,
Thomas

[admin]

Hi michael85, thank you for using FFAStrans and welcome to the forum!

Currently the monitor nodes does not support file and media variables. The reason for this is that the variables are populated AFTER a file has been picked up by the monitor. So as Thomas writes, you need to use the conditional nodes for this kind of logic. As to picking up files based on date, FFAStrans has currently no inbuilt support for this. But again, this is possible to achieve using the conditional nodes.
So take a look at the link posted by Thomas and see if you can work it out. If you still struggle with it, export your workflow attempt and post it here and we will help you

-steinar

[michael85]

Thank you for welcoming me and for your great support.
I have changed my tiny workflow as suggested and put a conditional node between the monitor node and the encoder node. See attached Screenshot and workflow.

Unfortunately it didn't work like expected.
I have put two files into my test input folder. One file with a duration of 00:59:30:00 and one file with a much shorter duration to test my workflow.
When I start the workflow, I can see in status monitor on node2 (conditional node) "reading source media properties". This status disappears after a few seconds without any error or entry in the status monitor, regardless of clip duration. Transcoding will not start.

I don't know know if it matters, my source files are every time mxf op1a.

Anybody an idea why my workflow stuck on the conditional node now?

[emcodem]

Hey Michael,

In the Conditional processor, checking the "Dispel" checkbox makes the job dissapear from job monitor in case the condition was not met. Please just start with "Dispel" uncecked, you can later turn it on when you have more experience and want to "hide" the jobs that failed the condition.
Your H264 node is set to execute only on "Error", but you check if your TC IS the one you are looking for. ThomasM suggested below that you check if the timecode is NOT the one you are looking for.

Easy solution: set the 264 node to execute on Success and alter your condition to check if the TC is NOT the one you are looking for.

Also, for fast testing, here 2 tipps:
-) use the rightclick-submit function for submitting files instead of using the watchfolder, its faster
-) you can check out the actual "value" of variables using a Populate Variables node and set s_success to the variable of interest. For this, insert a standalone, not connected Populate Variables node and submit a file to it, so you see the value of your variable on the Job Monitor. - this way you can ensure if ffastrans is seeing the same timecode as you expect it to do.

[momocampo]

Hello Michael and welcome here.
I think you have to do a test first to check a file with a 59:30:00 duration has really the exact duration otherwise all your files will process.
Maybe you can look at the picture I did and create a generated text file to be sure of your duration_tc variable.(picture 1)
Just enter a path to the txt file and submit a file. You will be able to see if the duration is exactly 00:59:30:00.(picture2)

You can also have another point of view for your workflow. Instead of use a "=", you can use a < and a >. If you do taht, make sure you right click on H264 input connector and choose green.
So if the file found is smaller or larger than 59:30:00 it will be encoded to h264, otherwise no. (to avoid a lot of message in the status monitor, check the dispel job instance).(picture 3)
Hope it helps.
B.

[michael85]

Hello everyone and thank you for your outstanding help.
I have checked the Duration of my files in FFAStrans as described from you. And the result is really weird, which leads me to another problem.
FFAStrans detected a Duration of my (to be separated files) between 01:00:43:02 and 01:00:43:09 instead of 00:59:30:00. I have tested several files.
This happens only to files which were created by file ingest from my Video Server. All the other files (same format and codec), who comes from the post production (Adobe Premiere) has the same TC in FFAStrans as they were exported from Premiere.

I'am definitely sure, that my files from the Video Server has a proper Duration of 00:59:30:00. MediaInfo says the same to this Files. I have attached a report from MediaInfo of one of these Files.

Anyway, I tried the solution from momocampo (picture 3) and set the Conditional Node to:
IF %s_duration_tc% is smaller than (<) 01:00:43:00 AND
IF %s_duration_tc% is larger than (>) 01:00:43:10 THEN
H.264 Node = Execute on success

This ends in the result, that FFAStrans denys all files that I have tested:
File1 had of Duration of 00:02:08:08 --> Result: "1st evuluation of "%s_duration_tc%<01:00:43:00" is "00:02:08:08<01:00:43:00"
File2 had of Duration of 01:11:26:02 --> Result: "1st evuluation of "%s_duration_tc%<01:00:43:00" is "01:11:26:02<01:00:43:00"
File3 had of Duration of 01:00:43:03 --> Result: "1st evuluation of "%s_duration_tc%<01:00:43:00" is "01:00:43:03<01:00:43:00"

I dont understand why these 3 Files didn't met the condition No.1. because File 2 and File 3 are not smaller than 01:00:43:00.
I also do not understand why the Duration from my ingested files is far away from their real Duration. Is this a bug in FFAStrans or is there a problem with files from the video server?

Best regards
Michael

[admin]

Hi michael85,

I'm sorry but you can't compare TC like that. Essentially the conditional is based on comparing text and numbers, not special stuff like TC or other complex string notations. However, the solution is quite simple:

IF %f_duration% is smaller than (<) $tctosec("01:00:43:00") AND
IF %f_duration% is larger than (>) $tctosec("01:00:43:10") THEN

So you see you need use the duration in seconds (%f_duration%) and compare it to the $tctosec() function with the desired TC duration applied.
Hope this helps.

-steinar

[momocampo]

Hi all,
Oups, my mistake and I am so sorry. It doesn't make sense to compare tc values like that , true.
And more it is not "and" into the conditional but "or" (media can't be smaller AND larger !)....I realy need to rest
Anyway, look at the capture, it should be ok this time.
Sorry Boss, hope I won't be punished...

[emcodem]

I believe the duration difference that you find is because in Omneon Files which have unknown duration in the file header, ffmpeg/ffprobe estimates the duration from bitrate and file size which cannot be 100% accurate for LongGOP.

Mediainfo instead might be smart enough to check out the file footer and gives you the duration from Files metadata. You can access the Mediainfo duration like that:
$jsonget("%s_info_mediainfo%","media.track[0].Duration")
Not sure tough if mediainfo considers the MXF Metadata "Origin", so it is possible that that mediainfo still detects between 0 and 15 frames longer duration than Premiere and Omneon itself does, only 1 of 15 files would show accidently the correct duration in that case.

It might be possible for you to use the file size tough... If the files NOT to process are all the same duration and all of them from the same omneon version, they might have exactly the same size in bytes too...