Multiple, simultaneous workflows: low CPU usage and resulting in .avs files

[jwietholter]

I have multiple workflows setup to watch the same folder for input files.

When a new file is placed into the input folder, all of the workflows (there are 4 of them) begin their processing as they should. They go through all of their processing steps, but while encoding, the CPU usage hover around 13%-21%. The FFAStrans configuration has the CPU Roof set to 75% (default), but it never seems to get there, although it does balance encoding across all 8 cores on my computer. The rate of processing at the A/V Media stage is around 16-20 fps for each of the workflows.

When the processing is complete, one of the encodes succeeds (audio only MP3 file), but all of the others result in .avs files. As part of the Custom FFmpeg process, the status indicates "Checking AviSynth script...(TTL: ####)." The TTL numbers count down and when they reach zero, the processing is completed and the .avs files are written to the output.

Any help would be appreciated. Thanks!

[jwietholter]

If I run each workflow individually, 3 of the 4 complete their processing without issue. It appears that the one that is failing has some sort of issue with the encode settings which I am trying to debug.

EDIT: Also, it appears that when running one encode at a time, the CPU usage is around 90%-96%.

Thanks again for the help.

[admin]

FFAStrans tries to verify the source but that may take a long time if:

1. Several other processes accessing the same file.
2. If the source resides on a slow storage.
3. If the source file is VERY large.
4. In the case of source being an .avs file, it's got some heavy processor intensive filters.
5. If the source file is corrupt.

In your case I would guess it might be a combination of 1-4(?) but I don't know your particular setup so it's difficult to be precise.
Why don't you send me your workflows together with the custom encoders and I'll have a look. You'll find the workflows in the Workflows folder (duh!)... as .ini-files. The "Custom FFmpeg" presets are found under Processors\.enc_customff.

-steipal

[jwietholter]

I will send the Workflows and Processors. With regards to the specifics:

1 - All four workflows are accessing the same file, but there is nothing else operating on the file at the same time. We're not playing it back, copying it, or anything like that. It is at rest.
2 - It's not on an SSD, but the file does reside on an internal spinning disk drive connected via SATA.
3 - The source file is a 217MB AVCHD file.
4 - It does have filters, such as Channel Mapper, Resize, Deinterlace and FPS. These vary from profile to profile.
5 - This may be true. I will try with another file.

Thanks!

[admin]

You have some faulty settings in your custom "SD Widescreen MPEG-2" encoder:
First there is a misspelling in the audio codec. You have "lipmp3lame". The correct should be "libmp3lame".
The video option "-ilme" is invalid. Try replacing "-ilme -s 720x480 -aspect 16:9" with "-flags +ilme -vf scale=720:480:interl=1,setdar=16:9".

That said, I still think it's weird your setup fail when running all at the same time. Is it just this one 217MB AVCHD file you have problems with? Also, I could not find any filters other than the "Channel Mapper"-filter. Did you leave something out?

Anyway, your workflows will benefit much from the upcoming 0.6.0 version of FFAStrans as you only would need to watch and decode the files once and then branch it to 4 different encoders:

workflow.png (14.66 KiB) Viewed 14016 times

-steipal

[jwietholter]

I knew that something was out of place in the MPEG-2 encoder. I've been trying some different settings and that was just my most recent, albeit quick and dirty attempt at some fixes - with obvious problems.

Let me make those adjustments and see how it performs with a different source file.

[jwietholter]

Adjustments made, different source file being encoded. This time it is a much longer (1+ hour) file.

I'm encountering the same performance issues as before (e.g. CPU Usage is only around 12%-20% and the workflows are each processing at the A/V Media stage at around 13fps). This is on a 3.4GHz Intel Core i7 w/16GB RAM running Windows 7 Professional. The system drive is an SSD, the working drive is a 7200RPM internal SATA drive.

I tested the same files on another computer using the internal drive as the workroot and the source drive and it was utilizing 100% of the disk. So that looks to be the problem.

Besides the obvious step of separating the workroot from the source drive, are there any other performance enhancement steps that I can take for a single source file going through four different encoders?

[admin]

Well, what you could do is separate the decoding + filtering part of your workflow from the the encoding part and make sure last step of the new workflow is the "Channel Mapper". Now you can set up 4 new monitors + encoding that look for .avs files in the work folder of your new workflow. In order for that to work you have to make sure FFAStrans doesn't clean up after decoding and filtering. You can do this under the "Maintenance"-tab in the workflow setup. With this approach you can somewhat create a similar workflow as you would do with the next release of FFAStrans. I haven't tested that particular scenario but it should work in theory.

Other than that I can't think of anything you might do with the current version in order to improve speed.

-steipal

[Mike]

Hello, man! Use your great soft more than month, its better than code monsters like carbon coder and mediaencoder.
But I need help with new task. Convert any video's to mpeg2. I need simply mpeg2 with low or non compressed sound.
I build new graph, and have some troubles with custom ffmpeg.
Please give me right way! Thanks a lot!

-flags +ilme -vf scale=720:5760:interl=1,setdar=16:9

[Mike]

Of course 720-576. But problem still.