API pause/resume PUT split index issue

[emcodem]

Merry Christmas steinar, it is ok if you don't answer very quick to this;-)

While implementing the Pause/Resume, i noticed that in the in the current version and in the beta, that there seems to be an issue regrading the split id. You can check it out using the most current Version of the Webui but i also invested lots of time e.g. using postman. Here some example PUT call:

Code: Select all

[[23:46:47.680]] [LOG]    http://localhost:65445/api/json/v1/jobs/20181223-234623-369-AD8B9C3895ED
[[23:46:47.682]] [LOG]    { action: 'resume', split: 0 }

The problem is, when we have a split, e.g. a workflow starting with "populate variables" and then splitting up into 2 sleep (60) modules. When there are 2 splits and setting the "split" element of the PUT body to 0 or 1, it seems like one of the both just does not work, but randomly it is 0 or 1 that don't work. Also it is not clear to me how to identify from the API get Jobs call which split number belongs to which split entry in the job. Currently i use the array index of the splits because there is no splitid in the "steps":

Code: Select all

{"discovery":"http://desktop-uifqt86:65445/api/json/v1","jobs":[{"job_id":"20181223-235154-318-CCBB2F21B995","job_start":"2018\/12\/23 23:51:55","file":"D:\\ExtendLastFrame.xml","wf_name":"XDCAMHD","splits":[{"steps":"2 \/ 2","processor":"Hold","status":"Resting workflow...","node":"DESKTOP-UIFQT86","progress":"50"},{"steps":"2 \/ 2","processor":"Hold","status":"Resting workflow...","node":"DESKTOP-UIFQT86","progress":"50"}]}]}
 

Also, please note that i had a huge problem working with the PUT. Unfortunately i needed to work on it about 6 hours or more, it was expected as a 30 minutes thing Don't ask, some node.js middleware was inserting a BOM byte which is not allowed for JSON and i was not able to find out why or prevent it. My workaround was stopping the client from doing API calls but do API calls only on the server and implement special support on the server for it. If you have a chance, please only use GET and POST in future, the server and client libraries api consumers use may behave unexpected and strange for other methods.

Please enjoy christmas days (without working)
cheers!
emcodem

[admin]

The idea when I started to work on the API was to use the HTTP methods best fit for the operation. Because that's what the methods are there for, at least so I thought. However, if it raises issues and adds complexibility the I might want to reconsider. At the same time I don't want to change the API just because some middleware does strange things. So, it would be nice to know "whos fault" it is....

-steinar

[emcodem]

Totally agreed that you don't need to change any existing stuff just because some programmer doesnt know his tools
Here "who's fault" it was, look at the comment in this code:

Code: Select all

app.use('/proxy', proxy("http://"+global.config.STATIC_API_HOST+":"+global.config.STATIC_API_PORT,{
      parseReqBody: false,
      reqBodyEncoding: null,
      reqAsBuffer: true,
        proxyReqBodyDecorator: function(bodyContent, srcReq) {
        bodyContent=(""+srcReq.body) 
       //the ""+ is important here, it works around that node adds strange bytes to the request body, looks like BOM but isn't
       //we actually want the body to be forwarded unmodified
        return bodyContent;
      }
    }));

this is all it needs in node.js to implement a "real" proxy, so in my case instead of calling from the WebUI client's browser directly to the ffastans engine, all API calls go to my webserver in the exact format as ffastrans api wants it but with a prefix /proxy.
This basically works, but without the line bodyContent=(""+srcReq.body), the node middleware "Bodyparser" (which is heavily used by most node users) was modifying the body, adding some bytes at the start.

The engineer in me wants to ask you, shouldn't a PUT method more be used to create new jobs instead of modifying existing ones?

But the stuff with the PUT was just a sidenote, so back to the topic: are you able to confirm there is an issue with sending pause/resume to a job that has splits? - I temporarily disabled pause/resume functions on the webui because i wasn't able to get it working as expected

Cheers!
emcodem

[admin]

Yes, there is an issue with the pause/resume API. It works in the monitor app but not in the API. Thanks for finding this bug! I will fix it post the beta so you can test.

-steinar

[admin]

Ok, so I have introduced a "split_id" key to be used to define what job to bause and resume. I have also updated the API so that you can use the same POST method for aborting job instances. Look at the new docs on the discovery page. Please let me know how that works out.

-steinar

EDIT: The method is of course still PUT, not POST as I wrote. Referring to the HTTP/1.1 RFC specs: https://www.w3.org/Protocols/rfc2616/rf ... tml#sec9.6
PUT should be used to update an existing resource. So I think we have the correct use of PUT...?

[emcodem]

You are so right about the PUT, i just quickly read some definition and misinterpreted that it can be used for modifying existing resources but it shall - if the resource does not exist, create a new one. However, it does not make a big difference anymore...

Thanks a lot for the quick reaction!
I just tried the newest beta and i can confirm it works using the new split_id. Moreover, knowing how a splitid is built, it also works in ffastrans 0.9.3 so i was able to enable the pause/resume feature again. Thanks to your help!

We should close the topic now, i know, but i also know you read this and i don't want to span the forum with API related posts, so please allow the question:
Are you aware about the abort method is currently accepting only a jobid but no splitid. It aborts just all splits - which is different from how it works in the native job monitor.
But this is just a minor thing. I guess the most urgent needed change in the Api is the introducion of job and split ids everywhere -e.g. in history jobs, i am not able to determine a difference for splits that started/ended at the same time - so there is no chance for me to show both at the same time on the UI.