Advanced Logging
Posted: Tue Jan 01, 2019 3:46 pm
Hey steinar,
as promised, here some thoughts about logging.
What we currently have is already pretty good from my perspective. This is:
Time, Processor, Host, PID, Message
But looking at the logs, most information seems to be internal stuff, not relevant to the user (e.g. Executing ini, sys, Registering abort check etc... Also lots of internal ffmpeg commands that end with "at least one source file must be specified".
To come around and allow better filtering on a log viewer, i suggest to add these fields:
-) Severity (Info, Warn, Error, Debug, Trace) - where Trace includes internal stuff like "Executing ini"
-) Source (Internal, Processor STDOUT, Processor STDERR)
More interesting is the logged content. I would suggest that each Processor logs
-) it's input and output variables as INFO,
-) executed CMD (especially the ffmpeg command of interest for copy/paste manual usage) and their STDOUT,STDERR combined or using the Source specification above as INFO
-) executed processes exit code
Moreover, there should be the option to get logs via the API i refer to the log that is written when enabling the "write log to" option in a workflow and that worfklow logging is enabled by default.
To help parsing the logs i just need a way to find out where a log message ends, e.g. ffmpeg commands are just some text line in the log, not parseable. So currently, when i encounter such an unparseable line, i just presume that the latest parseable line's data (Time,Processor...) can be applied to the current line. This would only be possible when there is an "end of log msg" identifier, typically newline. So in the end each and every log msg value would need to print something different instead of newline into the log, this way each and every log entry should be parseable. Sure the newlines would need to be restored in the log viewer.
Cheers,
emcodem
as promised, here some thoughts about logging.
What we currently have is already pretty good from my perspective. This is:
Time, Processor, Host, PID, Message
But looking at the logs, most information seems to be internal stuff, not relevant to the user (e.g. Executing ini, sys, Registering abort check etc... Also lots of internal ffmpeg commands that end with "at least one source file must be specified".
To come around and allow better filtering on a log viewer, i suggest to add these fields:
-) Severity (Info, Warn, Error, Debug, Trace) - where Trace includes internal stuff like "Executing ini"
-) Source (Internal, Processor STDOUT, Processor STDERR)
More interesting is the logged content. I would suggest that each Processor logs
-) it's input and output variables as INFO,
-) executed CMD (especially the ffmpeg command of interest for copy/paste manual usage) and their STDOUT,STDERR combined or using the Source specification above as INFO
-) executed processes exit code
Moreover, there should be the option to get logs via the API i refer to the log that is written when enabling the "write log to" option in a workflow and that worfklow logging is enabled by default.
To help parsing the logs i just need a way to find out where a log message ends, e.g. ffmpeg commands are just some text line in the log, not parseable. So currently, when i encounter such an unparseable line, i just presume that the latest parseable line's data (Time,Processor...) can be applied to the current line. This would only be possible when there is an "end of log msg" identifier, typically newline. So in the end each and every log msg value would need to print something different instead of newline into the log, this way each and every log entry should be parseable. Sure the newlines would need to be restored in the log viewer.
Cheers,
emcodem
The work has begun and already I have a much better log than what's in the current version. I will let you know when I have something to present in the betas.