Using Logs

[kibi]

I need logging of workflow jobs for every video file going through FFASTrans.
That's why I turned on logging to the directory for each workflow, and point it to different dirs.

Could you please describe, how to locate a log file for a specific video file which FFASTrans starts to process?
There are a lot of files like this: YYYYMMDD-NNNNNN-NNN-FFFFFFFFFFFF.txt and it seems that there is more complicated logic than "one video file - one log file".

I want to send by mail the resulting log for every video file, gone through the FFASTrans.

[admin]

Hi Alexander

You can contstruct the log file name file name using the "%s_job_id%" variable. And you probably know the path

-steinar

[kibi]

Hi, Steinar!

Unfortunately it's impossible to use templates in the directory name for log files:

21-11-2016 13-00-32.png (12.59 KiB) Viewed 10580 times

I would like to send a united bunch of logs for exact video file, coming through the FFASTRANS. At present it's hard to locate the log file for each video file. Log's do not contain information about file name. Only jobs ID. Usually Logs do not contain the initial file name. This is the excerpt form log, saying that a new file was picked up without information about it's name (!):

Code: Select all

==> 2016-11-14 21:42:41.796 on Folder@RENDER-FFT-018, PID: 5052 -> Ticketfile is "F:\FFASTrans_stage2\Processors\.cache\tickets\x20161114-214241-124-E60FA6333555_20160609123554~glob~RENDER-FFT-018~20161114-214241-015-72E001323ED5.ini"
==> 2016-11-14 21:42:41.811 on Folder@RENDER-FFT-018, PID: 5052 -> Getting top storage...
==> 2016-11-14 21:42:41.811 on Folder@RENDER-FFT-018, PID: 5052 -> Creating work dir: F:\FFASTrans_stage2\.work_media_cache\20160609123554\20161114-214241-015-72E001323ED5
==> 2016-11-14 21:42:41.858 on Folder@RENDER-FFT-018, PID: 5052 -> Registering abort check...
==> 2016-11-14 21:42:41.874 on Folder@RENDER-FFT-018, PID: 5052 -> Executing ini: "mon_folder 20160504-201547-998-40DB3892D785"
==> 2016-11-14 21:42:41.874 on Folder@RENDER-FFT-018, PID: 5052 -> Executing sys: "mon_folder 20160504-201547-998-40DB3892D785"
==> 2016-11-14 21:42:41.905 on Folder@RENDER-FFT-018, PID: 5052 -> New file picked up by: "RENDER-FFT-018"
==> 2016-11-14 21:42:41.936 on Folder@RENDER-FFT-018, PID: 5052 -> ProcessorEnd_Routine(mon_folder 20160504-201547-998-40DB3892D785)
==> 2016-11-14 21:42:41.968 on Folder@RENDER-FFT-018, PID: 5052 -> Ending process...

In case of successful process, the last block in log contains target name:

Code: Select all

==> 2016-11-14 22:13:50.687 on Command executor@RENDER-FFT-021, PID: 7748 -> =================================================================================================================================================================================
==> 2016-11-14 22:13:51.859 on Command executor@RENDER-FFT-021, PID: 3928 -> Ticketfile is "F:\FFASTrans_stage2\Processors\.cache\tickets\x20161114-221351-249-77362C55BE24_20160609123554~glob~RENDER-FFT-021~20161114-214241-015-72E001323ED5.ini"
==> 2016-11-14 22:13:51.859 on Command executor@RENDER-FFT-021, PID: 3928 -> Source is \\172.17.21.8\web\video\main\HiBitrate\HD\RUNDOWN-8942373.mp4
==> 2016-11-14 22:13:51.874 on Command executor@RENDER-FFT-021, PID: 3928 -> Creating work dir: F:\FFASTrans_stage2\.work_media_cache\20160609123554\20161114-214241-015-72E001323ED5
==> 2016-11-14 22:13:51.906 on Command executor@RENDER-FFT-021, PID: 3928 -> Registering abort check...
==> 2016-11-14 22:13:51.921 on Command executor@RENDER-FFT-021, PID: 3928 -> ProcessorEnd_Routine(cmd_run 20160224-185139-889-B59BE20074EF)
==> 2016-11-14 22:13:51.937 on Command executor@RENDER-FFT-021, PID: 3928 -> Job completed without errors.
==> 2016-11-14 22:13:51.968 on Command executor@RENDER-FFT-021, PID: 3928 -> _EnumIfNewJob() with current queue:6
==> 2016-11-14 22:13:51.968 on Command executor@RENDER-FFT-021, PID: 3928 -> Ending process...
==> 2016-11-14 22:13:51.984 on Command executor@RENDER-FFT-021, PID: 3928 -> =================================================================================================================================================================================

Whereas failed, I have to only make guesses about what was the name of the source file.

And there are a lot of orphans log files like this:

Code: Select all

==> 2016-11-14 23:25:50.017 on Folder@RENDER-FFT-022, PID: 6636 -> Ticketfile is "F:\FFASTrans_stage2\Processors\.cache\tickets\x20161114-232549-532-43886F3C6554_20160609123554~glob~RENDER-FFT-022~20161114-214418-797-E40F98A3D259.ini"
==> 2016-11-14 23:25:50.017 on Folder@RENDER-FFT-022, PID: 6636 -> Getting top storage...
==> 2016-11-14 23:25:50.032 on Folder@RENDER-FFT-022, PID: 6636 -> Creating work dir: F:\FFASTrans_stage2\.work_media_cache\20160609123554\20161114-214418-797-E40F98A3D259
==> 2016-11-14 23:25:50.064 on Folder@RENDER-FFT-022, PID: 6636 -> Registering abort check...
==> 2016-11-14 23:25:50.079 on Folder@RENDER-FFT-022, PID: 6636 -> Executing ini: "mon_folder 20160504-201547-998-40DB3892D785"
==> 2016-11-14 23:25:50.079 on Folder@RENDER-FFT-022, PID: 6636 -> Executing sys: "mon_folder 20160504-201547-998-40DB3892D785"

It's not obvious, which log file belongs to which source video file. Usually I just filter it by size:

21-11-2016 13-19-29.png (97.58 KiB) Viewed 10580 times

[admin]

You can not use the variables like that on the log path. You need to construct the file path like this:

F:\Pipeline\Logs\AVS to MP4 (HD-HiBitrate)\%s_job_id%.txt

This string can be used in your workflow to localize the job log file.

Regarding log files; It's been a while since last time i gave it some attention and I know it need to be improved, but FFAStrans is a big project and time is a factor, so I have to chose my battles carefully

-steinar

[kibi]

I see...
Command executor node with following parameters should do what I want:

%comspec% /c "copy "F:\Pipeline\Logs\AVS to MP4 (HD-HiBitrate)\%s_job_id%.txt" "%s_original_path%\Processed\%s_original_name%_%s_original_ext%.log""

If it's possible, I can help you in the project.

[admin]

That's correct

If you want to contribute to the project then send me a private message on what you would like to do. Thanks!

-steinar