how to start a workflow without looking at existing files

Questions and answers on how to get the most out of FFAStrans
megam
Posts: 12
Joined: Wed Dec 26, 2018 8:56 pm

Re: how to start a workflow without looking at existing files

Post by megam »

admin, emcodem
emcodem wrote: Thu Dec 27, 2018 11:22 am Ah, that's easy then, "rebuild history" does exactly the opposite from what you expect: it checks for all files in the folder and marks them as alraedy processed. I think what you want is to also check the "forget missing files" option.

Let me know if there are any more questions left!
Yes it is, in a friend, if for some reason there will be no file in the TestOut folder when you click the rebuild history button or better yet, it will automatically transcode the missing files. This is true where you need to transcode a large number of files 1000-3000 or even more with time. But in this version of FFAStrans, rebuild history does not work.
And yet, in my workflow, the sources should not be moved to another folder, this is the reason for using the Rebuild history button. We want to introduce FFAStrans in the automation of Imagine Communications (HARRIS).
admin
Site Admin
Posts: 1687
Joined: Sat Feb 08, 2014 10:39 pm

Re: how to start a workflow without looking at existing files

Post by admin »

What version of FFAStrans are you using? I'm pretty sure rebuilding history works in the current version. Just tested it... Are you sure you have understood the concept of rebuilding history cache? The monitor node have no clue about what happens to the files once the picked up file is delivered to the next connecting node. So, rebuilding history has no effect on the files on your output folder.

-steinar
megam
Posts: 12
Joined: Wed Dec 26, 2018 8:56 pm

Re: how to start a workflow without looking at existing files

Post by megam »

admin wrote: Fri Dec 28, 2018 9:42 pm What version of FFAStrans are you using? I'm pretty sure rebuilding history works in the current version. Just tested it... Are you sure you have understood the concept of rebuilding history cache? The monitor node have no clue about what happens to the files once the picked up file is delivered to the next connecting node. So, rebuilding history has no effect on the files on your output folder.

-steinar
Thanks for the quick response!

I attached my project. I only have the Clear Rebuild button.

version FFAStrans 0.9.3
Attachments
Hi-res-to-Low-res.xml
Workflows
(7.71 KiB) Downloaded 500 times
About FAStrans.png
About FAStrans.png
rrrrr.PNG (13.36 KiB) Viewed 11533 times
momocampo
Posts: 595
Joined: Thu Jun 08, 2017 12:36 pm
Location: France-Paris

Re: how to start a workflow without looking at existing files

Post by momocampo »

Hello,
Well, I saw workflow and I don't see what is wrong. Clic on rebuild history if you want and you will avoid to encode the mxf files already present in your monitor folder, FFastrans will encode only the new ones after the rebuild history.
Is it what you want to do?

Thanks
Cheers.
Benjamin
megam
Posts: 12
Joined: Wed Dec 26, 2018 8:56 pm

Re: how to start a workflow without looking at existing files

Post by megam »

momocampo wrote: Mon Dec 31, 2018 12:18 pm Hello,
Well, I saw workflow and I don't see what is wrong. Clic on rebuild history if you want and you will avoid to encode the mxf files already present in your monitor folder, FFastrans will encode only the new ones after the rebuild history.
Is it what you want to do?

Thanks
Cheers.
Benjamin
I check on another computer. I recorded a video of how I do it all.
https://fex.net/864645068997

OS: Windows 10 Pro x64 1809

Thanks!
momocampo
Posts: 595
Joined: Thu Jun 08, 2017 12:36 pm
Location: France-Paris

Re: how to start a workflow without looking at existing files

Post by momocampo »

Ok I think I know why you have the problem. the rebuild history is about the SOURCE FOLDER "TEST_IN" in your case.In your video, you delete files from Destination folder...
I think you don't understand the difference between clear cache and rebuild history.
**If you want to encode again your file in source folder --> you must clear cache (or check "forget missing file" in monitor folder")--> the encoded files will overwrite old ones (into destination folder)if you checked "overwrite" in monitor folder.
**If you want too avoid encode a lot of file already into your source folder, rebuild history will write in cache these files -> No encode to destination folder until new files into source folder.
I hope your understand now the difference. Otherwise, please tell us exactly what do you want to do.
Cheers.

Benjamin
megam
Posts: 12
Joined: Wed Dec 26, 2018 8:56 pm

Re: how to start a workflow without looking at existing files

Post by megam »

momocampo

OK, what I need:
I have a task to make actual proxy copy for sourse video * .mxf

Because if the network disappears or the source file is not of high quality, then we replace the file from the source folder, and we need to re-transcode this file.
Trublshuting: I delete the file from the destination folder and automatically FFAStrans begins to transcode this missing file.

FFAStrans should compare two folders source and destination, if one or more proxy copy files are missing, then start auto-transcode the missing files.
Is it realizable in FFAStrans?

sorry for my English)
Thanks!
admin
Site Admin
Posts: 1687
Joined: Sat Feb 08, 2014 10:39 pm

Re: how to start a workflow without looking at existing files

Post by admin »

megam, FFAStrans does not work that way, and even more, I can with pretty high confidence say that no workflow engine works that way. You have to think in the terms of signal flow; Once a signal (a file) leaves a processor node that processor node have no clue what happens next. This is logic by design and it's not a bug or bad functionality. So FFAStrans will never do what you ask and you have to solve your problem in a different way.

Is this a normal scenario or does it just happen sometimes?

-steinar
megam
Posts: 12
Joined: Wed Dec 26, 2018 8:56 pm

Re: how to start a workflow without looking at existing files

Post by megam »

admin wrote: Mon Dec 31, 2018 10:30 pm megam, FFAStrans does not work that way, and even more, I can with pretty high confidence say that no workflow engine works that way. You have to think in the terms of signal flow; Once a signal (a file) leaves a processor node that processor node have no clue what happens next. This is logic by design and it's not a bug or bad functionality. So FFAStrans will never do what you ask and you have to solve your problem in a different way.

Is this a normal scenario or does it just happen sometimes?

-steinar
This happens very rarely, I hope. But my task is to foresee this before introducing FFAStrans.
OK, in that case, how can I manually specify FFAStrans to transcode the desired file again? Transferring sources file is not possible.

Thanks!
admin
Site Admin
Posts: 1687
Joined: Sat Feb 08, 2014 10:39 pm

Re: how to start a workflow without looking at existing files

Post by admin »

Manual submitting of files can be done in the FFAStrans Workflow Manager. Just right-click a workflow and select "Submit file(s)". There's also a new web-interface available these days for managing jobs and submitting files: viewtopic.php?f=5&t=789

How big are the folders being monitored? Just asking because the next version will have the option to continously check for changig file size. This might help your case but because it's more resource demanding than normal monitoring it might not be a good option for very large folders (with subfolders).

-steinar
Post Reply