FFAStrans wiki

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
====== Plugin Processor development ======

Since Version 1.0, FFAStrans supports adding processors as a plugin. This allows everyone (who is able to) to create his own processors or to download existing processors that other users created.
Download an example processor to start with here: {{ :custom_proc:custom_node_example.zip |}} and extract it to \FFAStrans\Processors\plugin_nodes\custom_nodes - then restart the workflow editor and find the example node in "Others" category.

==== Plugin Deployment ====

Plugin processors are placed in the folder \Processors\plugin_nodes\custom_nodes. 
It is important that the name of the Folder matches the "name" property in the JSON definition file of a processor.
So the folder \Processors\plugin_nodes\custom_nodes can only contain folders, not files. Each subfolder represents one plugin processor. For information about the contents of each plugin subfolder, keep on reading.

{{gallery>:custom_proc:plugin_base_dir.png?400x400&800X800&lightbox}}

In this picture, we see the plugin base directory \Processors\plugin_nodes\custom_nodes. The 2 folders represent each one custom plugin.

{{gallery>:custom_proc:plugin.png?400x400&800X800&lightbox}}

In this picture we see the contents of an example plugin. The file style.css is not mandatory, it is just used by this processor's index.html. The folder /bin is mandatory, it must contain the script/executeable.

------------------------
==== Plugin architecture ====

A plugin processor consists of these relevant parts:
  * JSON definition file (node.json in plugin root folder)
  * HTML user interface (index.html in plugin root folder)
  * Processor script/executeable (in /bin directory of a plugin, name configured in JSON definition file)
  * HTML help file (help.html - next to index.html)

=== Loading / Lifetime ===
ffastrans.exe and processors.a3x search plugin processors node.json definition from the filesystem on program startup. This means, when a new processor is added, you need to close and restart ffastrans.exe but a running job would theoretically find new plugin processors with each and every process step.

index.html is loaded whenever you open the processor's gui in the workflow manager. 

The processors script/executeable is only executed when a job actually drives the processor by spawning a corresponding processors.a3x process along with a job ticket.

===JSON definition file===

<code>
{
        "name":"Custom Node Example",
        "category":"Others",
        "enabled":"1",
        "priority":"1",
        "description":"Example node",
        "guid": "312dccee-88e1-45b1-906c-9c1234724805",
	"script_interpreter": "",
	"proc_script":"processor.exe",
	"timeout_minutes": 120,
	"gui_window_height": 400,
        "version":"1.1"
}
</code>

  * name: display name of the processor in the workflow canvas, name must match the "base directory name" of the plugin within the plugins folder.
  * category: display category in the workflow canvas. possible values: TODO
  * enabled: when set to 0, processor is not shown in the processor menu in workflow canvas
  * priority: 1 is default, when set to -1, the node is not counted as a running job. Used e.g. for sleeping and waiting processes while allowing other jobs to process instead of blocking the que
  * description: tooltip of the processor icon in workflow canvas
  * ** guid: must be unique for each custom plugin**, best use a guid generator. A workflow stores the plugin based on it's guid, so the guid should not ne changed after plugin installation.
  * script_interpreter: only needed if the "proc_script" is not an .exe file. script_interpreter must not be a path but can also be a full set of commands; you can write here whatever you would write on a command line that's needed to execute your script file, e.g. "c:\strawberry\perl.exe" -d 
  * proc_script: filename of the .exe or script, "relative" from the /bin directory of the plugin. Just the filename goes here, not more.
  * timeout_minutes: a running processor will be killed by ffastrans when the timeout exceeded
  * gui_window_height: the processor gui height in the workflow canvas
  * version: the current version of this processor. Developers need to raise this number with every release and it can and should be displayed on the index.html gui


----------------------------------------------

==== Plugin Development ====

Please note that it is best for starting a new processor, to use an existing processor as a template. You can start with the example processor {{ :custom_proc:custom_node_example.zip |}}. If you want to go with another language than Csharp for the processor.exe, please ask in the forum for another example or read on this guide.


=== Overview of execution ===

After a Custom processor was saved and configured in a workflow and a job was started, the custom processor is executed this way:
  - ffastrans starts a cmd.exe,  building a commandline string the following way **"script_interpreter" "proc_script" "%temp_path_to_jobjson%" "%s_source%"**
  - The processor may read the jobjson including input variables and maybe create the outputs.json file in order to pass variables back to ffastrans
  - The processor may do something different, like transcoding or whatever the programmer of the processor intends it to do
  - FFastrans cheks for variables in outputs.json and stores them so the following processor can use the new variable values

=== Help File (help.html) ===

This is the page that opens when a user clicks the question mark bottom left in the processor. 

To copy the style of the existing help pages of other processors, you can use this example:

<code>
<html>
<head>
<style>
html *
{
  font-family: Calibri;
  font-size: 14px;
}
</style>
</head>
<body>
This processor node will put the job instance in pause until one of the following criterias are met:<br/>
<br/><br/>
<b>Sleep</b>
<br/>
Pauses the job instance in the amount of seconds specified in the "Time" input.<br/>
<br/>
Wait for files
Pauses the job instance until the file(s) in the list is present.<br/>
<b>One can specify any number of files in the list.</b> 
</body>
</html>
</code>


=== User Interface (index.html) ===


Note that the content of the HTML page itself is not anyhow predefined but you should create a look and feel to match all other existing processor GUI's. All relevant that the html page does is to save and restore a simple JSON structure containing Inputs and Outputs which will be passed to a processor.exe at execution time.

This index.html is used to create the User interface of the processor in the workflow editor ONLY, it is not involved in the actual execution of the processor in a running job. You can create a custom web page that accepts inputs from the user. The input names and values of the saved processor instance in your workflow are passed using a temp JSON file to the processor.exe. 

{{gallery>:custom_proc:custom_proc.png?400x400&lightbox}}

In the picture above, you see the area in green where your index.html is rendered using IE11. You cannot change the area outside as those buttons are created by FFAStrans host. The only thing you can influence is the display height by altering the plugin definition json. 

The example processor that you can download above contains examples how to use all possible functions and inteactions between the FFAStrans Engine and the index.html. 

When a user opens the processor GUI on the workflow editor, FFAStrans will create an Embedded Browser which is Internet Explorer 11 so always develop and test in IE 11. 

Note that on some Systems, it is needed to force the document to run in IE11 mode by having these 2 lines at the top of index.html. If you do not add this, the javascript base64 encoder and decoder atob and btoa migth not be available and your plugin might randomly not work on a recent Windows Version.
<code>
<!DOCTYPE html>
<meta http-equiv="X-UA-Compatible" content="IE=11" /> 
</code>

For interacting with the embedded Browser, FFAStrans host "executes" or evaluates javascript functions with predefined names and passes Base64 encoded data as argument to the functions. See the example processor for more description. The opening procedure is like that:
    - Create IE11 and load index.html
    - Call function named ffas_init and pass the processor name to it  
    - Call function ffas_load_preset and pass the last saved processor configuration to it
    - When user presses the save preset button, call the function ffas_save_preset() - this function must collect all data to save in a special JSON structue, see below.

== javascript function  ffas_load_preset(json_object) ==

FFAStrans Host calls this function on page load to restore the last configuration or when the user loads a preset. 
The function must loop through the values of the JSON file and restore the values of the corresponding input and output "textboxes" (or other html elements). 

Parameter: A JSON structure containing Inputs and Outputs.
Note that inputs and outputs is not related to HTML, it is just used to let the processor.exe later know what it can use as input variable and what it needs to set as output. 
<code>
{
  "inputs":[{"id"="some custom id", "value"="last_saved_value"}], 
  "outputs":[{"id":"some custom id","value":"some ffastrans variable name"}]
}
</code>


== javascript function  ffas_save_preset() ==

The Host calls this function when the user hits the OK button or saves the preset in the preset menu. This function is expected to collect the inputs and outputs from the html page (e.g. using document.getElementById) and create the same JSON structure as described above in the ffas_load_preset function. 
Don't forget to Base64 encode before returning the value.

== javascript function  ffas_init(displayname) ==

The Host calls this when the page was loaded and supplies the displayname as parameter. You can use this as a replacement for body.onload event if you like but it is not mandatory to implement this function.


== javascript function  ffas_folder_selected(selected_folder,connected_input_id) ==

Parameters are as usual base64 encoded. 
The Host calls this when a folder was selected. Folderbrowser can be opened using a special HTML element name, see below. You need to decode the selected_folder argument from base64 and set the value of the connected_input_id element accordingly (or do whatever else you want to do with the selected folder string)

== javascript function  ffas_file_selected(selected_folder,connected_input_id) ==

Same as folder_selected but for input with name "open_filebrowser". 

== Special HTML Elements ==

In order to open dialogs that are not javascript native like a filebrowser, you have to give a HTML Element a special name, e.g. "open_vars". You do not need to supply the onclick event, FFAStrans listens automatically for all click events of all elements and checks if it knows the name of the clicked element. \\
Additional Parameters are handed over using the HTML5 "data-" Attribute. 

  

Open a folderbrowser:
<code>
<input name="open_folderbrowser" type="submit" value="..." data-parent="path"/>
</code>
  * "data-parent" specifies the ID of the HTML Input box where the chosen path should go to
  * When a Folder was Chosen, the javascript function ffas_folder_selected will be called

Open a filebrowser: 
<code>
<input name="open_filebrowser" type="submit" value="..." data-parent="path"/>
</code>

  * "data-parent" if present, specifies the ID of the HTML Input box where the "starting path" should be read from

Open a variable selector window:
<code>
<input width="2px" type="submit" value="<" name="open_vars" data-parent="output_files_1" data-user_vars_only="true"/>
</code>

  * "data-parent" if present, specifies the ID of the HTML Input Box where the chosen value should go to.
  * "data-user_vars_only" if present, values allowed: true or false, specifies if you want to hide all other variables than user_vars. Use this for all HTML inputs where a user should just specify a single variable (e.g. which variable should be filled with the outcome of processor.exe)

=== The processor (processor.exe) ===

Please note that the word "processor.exe" on this page is just a placeholder for whatever you configure in the JSON definition file as "proc_script". It can be anything that can be any kind of program that can be executed on commandline. 

For example to execute the powershell script "processor.ps1", you need to configure the Plugin definition and set 

    * "script_interpreter": "powershell.exe -ExecutionPolicy Unrestricted -File "
    * "proc_script":"force_error.ps1"

This is because powershell needs the ExecutionPolicy to be overrided and also it needs the -File parameter when providing a ps1 file on commandline. 

Everything else you need to do is to place the force_error.ps1 script into the /bin directory of your custom processor.

FFAStrans will prepend the path to the job JSON file which contains information about inputs and outputs as first parameter and the full path to the current source file as second parameter. So a simple .bat Batch Processor file could access the source filename as %2.

** Reading Inputs from a processor **


grab the first commandline argument, it is the path to a jobjson, e.g. f:\.ffastrans_work_root\WorkflowGUID\JobGuid\Jobguid...plugin_htmlgui.inputs.json

Use a JSON parsing library to parse the contents of the file into a JSON object and look for 
 - processor_output_filepath
 - proc_data

The proc_data.inputs array will contain a list of userinputs. As the id's of the inputs were created on the index.html, the processor script has to be synchronized with the index.html and KNOW about what to do with which input id.

The processor_output_filepath should contain the path to the expected file containing output variable values, e.g. 
<code>
  "proc_data": {
    "outputs": [
      {
        "id": "testoutput",
        "value": "Choose a ffastrans variable, processor needs to set the variable by element id",
        "data": ""
      },
      {
        "id": "testoutput2",
        "value": "",
        "data": ""
      }
    ],
    "inputs": [
      {
        "id": "testinput",
        "value": "This is passed to processor"
      }
    ]
  }
  
</code>

** Set Outputs from a processor **

Setting outputs is optional. If you develop a processor that is not intended to deliver any output back to ffastrans, thats ok and you do not have to read on ;-)

The most simple output a processor can set is it's return code. If your script or exe returns any other exit code than 0, ffastrans automatically assumes a failure. 

If your processor shall be able to set some variables in the workflow, it must create a copy of the jobJSON that it got as input (see above) and enrich the proc_data.output array by the values that it wants to set. When the processor has finished its work, it should write the enrichted JobJSON to the path that was seen in "processor_output_filepath".

 
There are 2 different types of outputs: 
  * custom outputs that were defined in index.html
     * these are already contained in the jobJSON which the processor.exe did get on input. All the processor needs to do is to fill the (already existing but empty) "data" property. 
     * The id and value property have been set by the user on the GUI and should not be changed. The id will contain the name of the HTML element which was created custom on the index.html page 
     * The value property will contain the name of a valid user_variable name. The processor should not change both, it is expected to just set the data property. 
  * predefined outputs that let you set some status within the FFAStrans engine: 
    * s_source
      * this can be used to override the current source file (in case the processor created a new file e.g. transcoding)
      * **Note:** Setting s_source to an empty value causes the job to end immediately and dissapear from the job monitor. All cache files and records are deleted by ffastrans in that case.
    * s_success
      * this can be used to set a status message for the job monitoring
    * s_job_error_msg
      * setting this will cause the FFAStrans host to go the "error" path in the workflow or end the job with error

Usually outputs are already part of the job json because they are passed in from the processor GUI html. 
So the processor code does not have to "generate" a new object but instead it only sets the "data" attribute of existing outputs in the existing job json.

Take special note about the field names, "id" is optional as it is just the html id of the field on index.html GUI. FFAStrans only needs to know "value" and "data", where "value" has to be some existing variable (user var or predefined system var)

<code>
"proc_data": {
    "inputs": [],
    "outputs": [
      { 
        "id": "",
        "value": "s_source",
        "data": "C:\\test\\test.ts"
      }
    ]
  }
</code>


---------

<html>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</html>[[custom_proc_download: Download Additional Custom Processors|To Plugin Processor Download]]

<html>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</html>[[custom_proc:custom_processors|Back to top]]