Concatenate

[taner]

Hi Admin Team,

I have a question.
Concerning Concatenate.
I created a workflow where I submit a single file to a workflow and the workflow concatenates all files within the folder where the submitted file is located.
Using a bat-file which will be automatically created during the process.
It works as intended.
By the way: I have never given much attention to variables until FFAStrans came around
It makes FFAStrans a real powerhouse, not only for transcoding purposes but for every kind of workflow.
However.
I would like to create a workflow where I submit several files at once.
All files are within the same folder (source location is not always the same).
All submitted files shall be concatenated.
As far as I understand FFAStrans mechanism each submitted file is a separate job.
So, every submitted file is passing every node within a workflow-chain.
Every command would be executed several times.
So it would make no sense for example to use the hold node.

Have you an idea if such a workflow within a single workflow-chain would be possible?
I would prefer to avoid using a watchfolder and keep the workflow disabled.

It's absolutely definitely for sure not a life saving feature.
Apart from concatenating rather a starting point to make more fancy things out of it.

Best
Taner

[ThomasM]

Hi Taner,

just "mark" the folder "Occupied" with a simple .TXT-File

Monitor-Node -> Conditional-Node "IF $exists("%s_original_path%\OCCUPIED.TXT") = 0 THEN ->
TRUE: Generate-Text-Node -> Go On with your Workflow
FALSE: DISPEL JOB (Marker-Field lower-Left).

This should prevent from Multiple Calls of your Concetanation.


Just my 2 cents...

regards,
tom

[emcodem]

Hey Taner,

asides from ThomasM's magnificent 2 cents there are many ways how to deal with what you want to do.
Biggest question is how the workflow is triggered. In Webui i didnt yet add the "submit multiple files to a single job" altough i wanted to do this for a long time. So i guess you go for API submission?
Anyway, you got 2 choices: Submit the list of files or submit the folder and use something like a files find processor to generate the list of files.

One way to go for example:
1) 3rdparty logic creates ffconcat compatible list and submits this list as variable value
2) workflow writes ffconcat file to disk and kicks off ffmpeg commandline that processes ffconcat file

Where do you want to go exactly?

[taner]

Sorry for late response.
Thanks both for your input!
@emcodem: well you know, I'm lazy. Always looking for a one click solution.
I choose the price behind curtain 2.


Submitting a folder or rather all files within a folder or rather to be more specific just all files within a folder which have the desired specs is somehow what I do at the moment.
But submitting certain files within a folder at once to a single job without using any rigid/static logic would be great.

Best
Taner

[admin]

HI Taner,

Been thinking a bit on this and what we might be able to offer is the option to submit a bunch of files to a workflow variable as an array of files. Of course as one single job. With this array (list of files) you can do whatever you want. And the next version of FFAStrans will have more powerful functions to deal with this array, like create a ready concat file for you. It even support concat (version 1.0) input files by default. So hope this can improve the situation.
But the way FFAStrans core currently is written there is no way to support multiple source job files they way you want in one single job.
Note here the difference between source job files and a file list stored in an arbitrary workflow variable.

-steinar

[taner]

Thanks!

[emcodem]

Steinar just relates to the core features of ffastrans mentioning this "there is only one source" stuff. What he means is that there are some things like the original file variables that would not work if you submit a list of files to a workflow. But there is nothing that prevents you from submitting a list of files at all. You just have to take care about the list in your workflow somehow.

just all files within a folder which have the desired specs is somehow what I do at the moment.
But submitting certain files within a folder at once to a single job without using any rigid/static logic would be great.

OK so 2 things: what are "the desired specs" for you for example and
I am still not sure what you have in mind, are you looking for options that go with watchfolder now or you want to submit via API or Webinterface? Do you have a manual or automated workflow in mind?

[taner]

To be honest: both.
Manually submitting files and also by a watchfolder.
FFAStrans GUI.
But manually submitting would be more important.
At least for me.
Watchfolder a very-nice-to-have feature.

The "desired specs" stuff is related to my current workflow and actually not important.
At the moment the single submitted file just triggers creating a list with all files within the source folder identically to the file extension of the source file.
(for %%i in ("%s_original_path%\*.%s_original_ext%") do @echo file '%%i') > "%s_job_work%\concatenate\mylist.txt") bla bla

You know what would be great?
Concerning GoPro processor.
If one could submit GoPro files for stitching/concatenating purposes.
Same handling of spanned clips as GoPro processor.
That would be mega!

Best
Taner

[emcodem]

OK also to be honest: implementing a watchfolder is really problematic because without having a predefined way to determine the presence of all files, the number of options "when to start" a job is kind of indefinite. It is really hard to imagine how such watchfolder should work: do you wait like 5 minutes and check if no file is arriving anymore? Do you move the ready to process file into a subfolder before starting the process in order to free the watchfolder and make it ready for the next processing? Can the watchfolder really only be used by one user for one job within these 5 minutes - how do we guarantee that only one user uses the stuff concurrently? How do we find out which files belong together and which sorting options are there?
Most of these options are just quirksy....

Anyway, so for manual submission i can easily come up with something in the webgui if that is enough for you, just let me know.
For implementing your own watchfolder, you can use the files find plugin processor, it was designed for this kind of stuff. All you need is a way to trigger the actual start of the job, e.g. when all files are copied into the folder, write a text file there. Your workflow watches for text file only and uses files find to make a list of all files to process, sorts them and automatically generates the ffconcat list.
Did you give the files find plugin already a try?

[emcodem]

ok so here some example how i intended to solve one of the many concat usecases using the files find plugin processor.

It is designed to watch for "links" to folders that contain the source files. E.g. it is assumed that all source files are already statically resident in a directory. Then you create a link to that direcory in the watchfolder (e.g. hold alt and drag the folder into the watchfolder)

concatshortcut.png (29.89 KiB) Viewed 4404 times

This workflow only works when all source files have the same formats/audio config etc... (e.g. all are xdcamhd) otherwise the outputfile will be foobar.
For using the workflow, first install the plugin files find from here: https://ffastrans.com/wiki/lib/exe/fetc ... s_find.zip

emcodem_concat_same_formats.json

(8.98 KiB) Downloaded 248 times

For different source formats, we need another workflow that first transcodes each and every single file into the target format and only after that, execute the concat command, i'll probably post such an example later on.