Problem with large. AVI-files

[ThomasM]

Hi,

I encountered a problem transcoding large ( ~ 250... 350 GB). AVI-files. When attempting to transcode them with a custom ffastrans-node to prores, an error occures after part of transcoding saying " no moov atom found" or in another case sometimes "no audio or video found - file might be corrupt".

Then i tryed to decode them with a A/V-node and channel-mapper just before the prores-encoding. This failed also.

I tryed short AVI-files (~200 MB) and there is no problem with the same unchanged workflow.

The files all came from the same system. The AVI contains rgb24 images in 2k.

Ffastrans runs on a farm with two dedicated render-pc, ffastrans is started from a NAS, workfolder is on the same NAS (SYNOLOGY).

Is there maybe a problem with or in the encoding-step regarding the workfolder in terms of too large intermediate files?

Any help highly appreciated,
Regards,

Tom

[admin]

Hi Tom,

Are you running on 32 or 64 bit os?

-steinar

[ThomasM]

Hi Steinar, i am running the 64bit version.

Regards, Tom

[emcodem]

Hey Tom,

While a file is being encoded, can you see that it is being written ("anwachsend") in the cache folder you configured in ffastrans?

cheers,
emcodem

[ThomasM]

Hi emcodem,

thanks for your reply. I will check that tomorrow. All I can say now is, that the transcoding node starts and works to a certain point. The progression in the Monitor-Window was more than 1000 / 2500 (seconds). Are there any logs I can look in what happened?

btw: great How-To-postings!

regards,
tom

[emcodem]

Hey Thomas,

for the logs, please enable logging by right-clicking a workflow and in the special tab, check the "log to files" box...


cheers,
emcodem

[ThomasM]

Hey emcodem,

in the Working-Folder the file is growing.

I have now two files running, one 1159 seconds, the other 1381 seconds. Both AVI 2k 24Bit RGB. As described above, its all fine with smaller files (one second, same source, same size same bit-depth...).

Now the error occures on a CMD-executioner-node, which is only for copy ( %comspec% /c
"COPY "Z:\WATCHFOLDER\FA_Scanner\SCANNER-STUMM\RENAME\%s_original_name%_Archivdigitalisat.%s_original_ext%"
"Y:\WaitingDirectory\%s_original_name%\Temp"" ).

After this I get an error-message: Screenshot Number 1:

Screenshot (1).png (166.41 KiB) Viewed 10486 times

Then I try to render two files I already made in the morning. They both had no problems then. But by now, after the Error-Box, the Workflow seems to be broken:

Screenshot (2).png (169.71 KiB) Viewed 10486 times

As you can see in the FFAStrans-monitor, the first error on the Node (COPY AVI 2 WD) causes the rest of the encoding of the other file to exit on the Transcode-node. I was not aware of that, but in the end it is the same - I never had problems copying files that big to another NAS. The File-Size of 16FA100_Spenner is 267 GB. We have lots of those and already did many of them, but by now...

I will try Robocopy /Z /J instead of COPY. Will let you know what happens...

I attach the Workflow:

XX_fps_to25_fps_MUTE_SHARED_COPY.xml

(48.05 KiB) Downloaded 470 times

[emcodem]

Hey Thomas,

i see the error was with "exe_manager.exe" wich is part of ffastrans. I am not 100% sure but i believe you should restart not only ffastrans but the whole PC if you still have troubles (in order to make sure all ffastrans processes are re-spawned). This process should always run and is started with ffastrans. I guess if you now look into your task manager, you don't find any process named exe_manager.exe?

It is weird that exe_manager.exe is crashing because of just a copy command. Even the copy command itself should not crash. Typically such stuff only happens when some disturbing software is installed or you have a hardware error like defective RAM..

Two things:
1) if my assumption is right that actually the "copy" command is causing the troubles, you should find some interesting errors in the windows application error log

2) switching to robocopy or another tool might solve the problem but not the cause of the problem

[ThomasM]

Hi emcodem,

yes - I had to restart the whole farm. The funny thing is that the same copy-command is used from another machine delivering the same files to one of the NAS. So it is:

Scanner-PC with FFAStrans [COPY-Node] -> NAS002

Then the following workflow:

Renderer 1 and Renderer 2 (are running FFAStrans from NAS002 as a Farm) [COPY-node] -> NAS003.

The renderer-PC are only running Win10Pro and FFAStrans from the NAS. They are dedicated as render-Knights

[ThomasM]

...and this is what I get from large files if it even does not make it up to the COPY-node:

Screenshot (3).png (179.11 KiB) Viewed 10478 times

tAke a close look: it is the same files that I tried to convert this morning. All four files started to convert and had a status comparable to the executed workflows in the monitor above. Strange thing.

What Windows-log is worth looking at? I never done this before...

regards,

tom