Too Many Connectors

[Noqras]

Well, some time ago I managed to use the maximum number of processors, that was fixed and I can have tons of processors now. But just now as I was about to clean up my workflow to be a bit more sparing on the processors, it looks like I broke my workflow by having too many connectors between processors, see the attached screenshots.

I was able to create the workflow fine without error, but then I deleted a few processors and then tried to add a new connector and I get the Line 58774 error. If I deleted a processor and then try to delete another I get the second error Line 58504.

I found a workaround which is delete some connectors first, then I can delete processors or make some new connectors all I want without errors. The errors only occur after first having a huge amount of connectors and processors and then deleting a processor. If I get rid of some connectors first, its fine.

So I've come to the conclusion that my problem is too many connectors. As you can see my workflow is a little intense, so it actually doesn't surprise me.

[Noqras]

Nevermind, my workaround doesn't fully work, I think I'm going to need to delete all connectors and then maybe it will be normal again. If I delete a bunch of connectors now and then start delete processors, the remaining connectors start moving to random places.

[admin]

Hi Noqras,

Thanks for reporting! I don't suppose you could share a version of your broken workflow? It would be much easier to investigate

-steinar

[emcodem]

Aye,
i spent some time checking what you do with all the nodes and i am under the impression it would be better for you to get in touch with variables. This way you might reduce your workflow to 1-3 encoding nodes only.

For example, to get started, lets reduce all the "is there no audio" nodes and also reduce half of the encoding nodes:
1) take a deep breath, get a coffee, backup your existing workflow
2) insert one "is ther no audio" conditional in the main branch (before fps check)
3) insert 2 "populate" nodes in the main branch after the before inserted "is there no audio" check
4) in the success path (there is really NO audio), set 2 user variables:

Code: Select all

%s_lavfi% =  -f lavfi -i anullsrc=r=48000 -shortest 
%s_ac% = -ac 2

after3.png (7.18 KiB) Viewed 11448 times

5) in the error path's populate parocessor, set the s_lavfi and s_ac variables to "nothing"
6) both populate connector's outputs are connected to the "there is 1 or less audios processor

before1.png (10.84 KiB) Viewed 11454 times

after1.png (12.35 KiB) Viewed 11454 times

6) delete all ffmpeg nodes for the old "no audio" path and all "is there no audio" conditionaly before the encoding nodes
7) alter the old "there is audio" ffmpeg command from this

Code: Select all

%ComSpec% /c ""%s_ffmpegx64%" -i "%s_source%" -c:v dnxhd -vf "scale=1920x1080:in_range=tv:out_range=tv,fps=30000/1001,format=yuv422p" -vsync 1 -b:v 145M -c:a pcm_s24le -ac 2 -ar 48000 "M:\DPShare\Aurora\DNxHD Profile\output\QC Pass\%s_original_name%.mxf""

to this:

Code: Select all

%ComSpec% /c ""%s_ffmpegx64%" -i "%s_source%" %s_lavfi% -c:v dnxhd -vf "scale=1920x1080:in_range=tv:out_range=tv,fps=30000/1001,format=yuv422p" -vsync 1 -b:v 145M -c:a pcm_s24le %s_ac%  -ar 48000 "M:\DPShare\Aurora\DNxHD Profile\output\QC Pass\%s_original_name%.mxf""

You see? ...all i did was inserting the variables %s_lavfi% and %s_ac% into the existing "there is audio" ffmpeg command. If the variables are set, audio will be created, if not, we go with the contained audio..

before2.png (25.1 KiB) Viewed 11454 times

...i am pretty sure we could delete of most of encoding nodes by using further variables for fps, resolution and encoding settings (bitrate)

Also, it would be good if you insert a populate processor at start to set the output path, instead of writing the output path into each and every ffmpeg line:

%s_output_path% = M:\DPShare\Aurora\DNxHD Profile\output\QC Pass\

This way, if you need to change the path, you only need to change it at ONE processor, not at 50...

[admin]

Noqras, I found the cause of the first bug and it was a nasty one! It's fixed and I'm testing now. The second bug is probably related but I have not been able to reproduce that one. I will send you a patch for you to test when I'm done.

Thanks again for reporting!

-steinar

[Noqras]

Emcodem!

Thank-you so much for your help on this, yesterday I was thinking as this was happening that first of all there is probably a better way to do with involving variables and second of all I really just need to sit down and work that out.

I have removed 1/3 of the encode processors by leaving a lanczos flag on all processors thus being able to remove the redundant processors for 4k and 2k, then using variables for audio that you laid out will make this much more efficient.

[emcodem]

Oh you will be so much more mighty when you work with variables, i am looking forward to it. Let me know all your doubts and experiences.
If you have troubles, please spend some time on the workflow example thread, maybe it is of help: viewtopic.php?t=849

And also please don't forget to thank @admin for locating the bug.. he works his ass of for all of us!

[Noqras]

Steinar,

I tested your patch and it works perfectly! I don't get the first error anymore and it looks like the second error was more or less an extension of the first error as it would only ever happen after the first. Thank-you so much!

I am reducing my workflow down quite a bit now, but we now know that workflows have the potential of being huge without any issues now which is great.

Thanks!

[admin]

That's great news, Noqras

Again, thanks for reporting the bug!

-steinar

[Pogle]

Hello!
I also have a workflow with (too) many nodes. Now I cant delete nodes anymore and get the error message
Line 58504... Array variable has incorrect number of subscripts or subscript dimension range exceeded. I have no backup version with fewer nodes ):
Could you help me repairing my existing workflow? I deleted most of the connectors, but still get the error message when trying to delete nodes.
Could I send you my broken workflow or would this mentioned patch help me?

Thank you!