System exception - Illegal Instruction - Possible issue with resize module

[lazbert123]

Hi there,


I'm currently running latest version of FFastrans (1.3.0.2) but seem to be running into issues with the resize module where we didn't before. I've managed to work around it with a few of our workflows but a few require the resize module to complete the task. The error in the status monitor is "Validate: System exception - Illegal Instruction." It fails at the Channel Mapper module placed after the resize module (with a pad module in between for this example, others are resize -> channel mapper).

When looking in the json log it gives the following:

"code": "000000A9",
"error": "Validate: System exception - Illegal Instruction."

Illegal Instruction\r\n(v:\\.ffastrans_work_root\\20170616164537\\20220923-1152-2125-23d3-6e7a6db8ccec\\117-0-25~220923115341337~11664~20170315-163508-912-9c90ef77bdfc~avs_a_acmapper.avs, line 63)

Looking at line 63 in the .avs file refers to:

60 #node_name:Resize, id:20170616-164901-150-cffa401178ac
61 LoadCplugin("C:\Program Files\FFastrans\processors\AVS_plugins\JPSDR\x86\Release_W7\plugins_JPSDR.dll")
62 m_clip = Bob(m_clip, 0.0, 1.0)
63 m_clip = BicubicResizeMT(m_clip, 1440, 1080)



avs validation text file reported:

ffmpeg started on 2022-09-23 at 12:00:52

Report written to "v:\.ffastrans_work_root\20170616164537\20220923-1159-3093-1f2b-f8a04d8b559b\117-0-25~220923120051974~13372~20170315-163508-912-9c90ef77bdfc~avs_a_acmapper~avs_validation.txt"

Log level: 32

Command line:

"C:\\Program Files\\FFastrans\\processors\\FFMpeg\\x86\\ffmpeg.exe" -analyzeduration 33554432 -i "v:\\.ffastrans_work_root\\20170616164537\\20220923-1159-3093-1f2b-f8a04d8b559b\\117-0-25~220923120050943~13372~20170315-163508-912-9c90ef77bdfc~avs_a_acmapper.avs" -ss 0.5 -vframes 5 -hide_banner -nostdin -f null -

[avisynth @ 005af740] System exception - Illegal Instruction

(v:\.ffastrans_work_root\20170616164537\20220923-1159-3093-1f2b-f8a04d8b559b\117-0-25~220923120050943~13372~20170315-163508-912-9c90ef77bdfc~avs_a_acmapper.avs, line 63)

v:\.ffastrans_work_root\20170616164537\20220923-1159-3093-1f2b-f8a04d8b559b\117-0-25~220923120050943~13372~20170315-163508-912-9c90ef77bdfc~avs_a_acmapper.avs: Unknown error occurred



I've tried it with Lanczos etc. but that doesn't make any difference. Any ideas what this might be about?


Many thanks for all your efforts.

[FranceBB]

"System exception - Illegal Instruction" is Avisynth throwing an error when failing to run a C++ function.
At first I thought that given that the resizers made by Jean Philippe Scotto di Rinaldi are C++ and we're wrongly loading them as C plugins (i.e LoadCPlugin() instead of LoadPlugin() ), it could have been it, but I tested it and it's not the cause of the problem 'cause it still works:



I changed LoadCPlugin to LoadPlugin() anyway in the source code 'cause it was wrong, even if it's not the cause of the issue.

Out of curiosity, is there any particular reason why you're using the 32bit version of FFAStrans instead of the 64bit version?
I couldn't fail to notice this line in your log:

"C:\Program Files\FFastrans\processors\AVS_plugins\JPSDR\x86\Release_W7\plugins_JPSDR.dll"

That line is produced internally by the node using an internal variable which refers to arch and is populated on the fly, which means that either you started FFAStrans_x86.exe instead of FFAStrans.exe OR that you have the option that forces 32bit dependencies in your workflow.

Unless there's any specific reason for you to use the 32bit build, can you try to run the FFAStrans.exe and work with that instead of using FFAStrans_x86.exe?
Remember, the first is 64bit and the latter is 32bit.
I'd like to know if you have the same issue with that one.
I'm also gonna be honest with you, I don't really do many regression testing on 32bit only systems...

IF and only IF you have a 32bit only system and you have to run everything 32bit, can you try this build I made in April 2021 for my own personal 32bit system? https://we.tl/t-eQPF3duBUK
It works on my side when I use Avisynth on Windows XP x86 with Microsoft Updates 'til 2019 https://i.imgur.com/oyaY2AN.png
Last but not least, what OS are you running?

[lazbert123]

Hi @FranceBB,

Many thanks for the detailed response on this issue, it's much appreciated. Regarding the 32bit issues, that one's a fault on my part in that the "use 64bit Avisynth" was not ticked on the "A/V Media" node on that chain of the workflow. Not sure why it was unticked as now it's ticked by default, it's an old workflow that's been changed a lot in the past so I guess something happened there. I've been using the 64bit version of the FFastrans executable. Looking at the logs it does now appear to be using the x64 version of the plugin as you described:

LoadCplugin("C:\Program Files\FFastrans\processors\AVS_plugins\JPSDR\x64\Release_W7\plugins_JPSDR.dll")

Unfortunately I'm still having issues but now its just reporting:

"current_status": "Validate: Unknown error occured.",

The process fails at the "ChannelMapper" node consistently now but I'm struggling to find anything specific in the Export Log to why it's failing. The only thing I could see that might be relevant is:

"msg": "Error getting ffprobe data from \"v:\\.ffastrans_work_root\\20170616164537\\20221003-0018-4121-6e4e-fe6a5105c9f9\\113-0-16~221003001852699~11856~20170315-163508-912-9c90ef77bdfc~avs_a_acmapper.avs\""

Interestingly I tested bypassing the "ChannelMapper" node and the "Unknown error" got passed onto the Encoder node (in this case AVC-I) with a similar failure.

As a final test I again bypassed the "Resize" node (re-enabling the Channel Mapper) and the workflow processed the file successfully. I can only think this points at some issue around the "Resize" module. Is there anything you would suggest to isolate why this issue is occurring?

Further info: I'm running FFastrans on Windows Server 2019.

[FranceBB]

Uhmmmm if it's a very old workflow, something bad might have happened during the upgrade.
Could you send the workflow so that I can try to run it over here and confirm the issue?
I have Windows Server 2019 too so it will be easy to double check.
What I think might have happened is that some old nodes have not been upgraded correctly and therefore create a wrong AVS Script somehow.

You can attach the workflow here or just use wetransfer

[FranceBB]

Ok, so, I've got your workflow and it's fine in the sense that it completed successfully on my end as I tested it on the same servers I use in production:

test_resize1.png (180.71 KiB) Viewed 5926 times

Test_resize2.png (171.42 KiB) Viewed 5926 times

About the resizers, can you go to "C:\Program Files\FFastrans\processors\AVS_plugins\JPSDR\x64\" and replace it with this one https://we.tl/t-mDH93LuhBa and try again?
I compiled it with SSE (not even SSE2) so if it's an instruction set problem, it will work.

If it still doesn't work, would you mind installing the latest Microsoft C++ Redistributable from here: https://github.com/abbodi1406/vcredist/releases and test again?
Please test and let me know if it still doesn't work.

If it still doesn't work, would you mind trying this special build of FFAStrans I made? https://wetransfer.com/downloads/5929c3 ... 752/05563a
(for everyone else, please DO NOT use this build)

[lazbert123]

OK so I tried the new JPSDR .dll and that's now sprung into life again! That seems to have fixed it on a physical server and a VM running similar resize workflows and both Server 2019.

Apologies I don't have a great of knowledge around compiling but I'm guessing that as it only has SSE that isn't ideal. Would that point at an OS issue? Would it be worth upgrading the C++ redistributable anyway to see if that improves things with the old .dll?

[FranceBB]

OK so I tried the new JPSDR .dll and that's now sprung into life again! That seems to have fixed it on a physical server and a VM running similar resize workflows and both Server 2019.

Great!
I'm glad to see that it works fine.

I'm guessing that as it only has SSE that isn't ideal.

Yeah it will be slower, but it's a compromise we have to make as developers between speed and availability.
To be absolutely fair, Grandmaster wanted to keep things as compatible as possible by targeting SSE2 so that pretty much any CPU would run it, but I was the one pushing and pushing and pushing for AVX2... so... sorry for the inconvenience I caused.
By the way, this is a simple NNEDI3 + x265 benchmark to put things into context with a consumer i7 CPU:

SSE - 0.76fps
SSE2 - 1.23fps
SSSE3 - 1.33fps
SSE4.2 - 1.89fps
AVX - 2.09fps
AVX2 - 2.58fps

Would that point at an OS issue?

Nope. Well, I mean, not necessarily. Like, Windows Server 2019 supports up to AVX-512, Windows Server 2008 R2 supports up to AVX2, Windows Server 2003 supports up to SSE4.2 etc, however the CPU has to support those of course. In other words, if you have an SSE2 only CPU, for instance, you could be running Windows Server 2019, but it will still stay on SSE2.

Would it be worth upgrading the C++ redistributable anyway to see if that improves things with the old .dll?

Nope, it won't make the old DLL magically work 'cause it's pretty clear at this point that the CPU you're using doesn't physically have those registers, so it would fail. That being said, even though we do distribute the relative dlls needed in the same package, it's always a good thing to have the C++ Redistributable installed 'cause this way they're updated automatically by Windows Update, regardless of FFAStrans. I personally have them installed for a lot of different reasons, so I always encourage people to do the same.

Apologies I don't have a great of knowledge around compiling

No worries, it's not something that everyone should know unless they're really interested in the topic
By the way, if you're interested in the topic now, I'll explain it a bit by quoting myself from October 10th, 2019, 07:16 AM 'cause I'm lazy xD

SSE, SSE2, SSSE3, SSE4.1, SSE4.2, AVX, AVX2, AVX512 are CPU Instruction sets which in terms of a program refer to how the program has been compiled. A program can have manually written intrinsics (e.g parts written in assembly) or it can be written using C++ and then compiled.
To clarify this, there are low-level programming languages like Assembly and high-level programming languages like C++. When you write a program and then you compile it, it's up to the compiler to produce a code that the machine understands.
If you write it using a low-level programming language like Assembly, the code is gonna be very optimized as it's gonna be as close as possible to what the machine understands, however writing a full program in a low-level language is a nightmare, especially for complicated things, where it's easy to lose track of what you are doing and why you are doing it. As opposite, a high level programming language like C++ is easier to use, however it will be less optimized 'cause the compiler has to "guess" what you wanted to do and "bring" your code to something that a machine can understand. Generally, the idea is to use a high level programming language and then optimize the key parts by manually writing intrinsics, (which is not easy and it's far too advanced for me as I'm not able to do that and I only code using a high level programming language).
Let's suppose that a plugin is written in C++ only and then compiled using different assembly optimizations.
This way, it's gonna be the compiler that will try to understand what the programmer wanted to do and will try to optimize it for different assembly instructions, which won't be as fast as manually written intrinsics, but it will be faster than using no assembly optimizations at all.
Now, I know what you are thinking... you are basically thinking "fair enough, Frank, but what are those SSE, SSE2, SSE3 SSSE3, SSE4.1, SSE4.2, AVX ecc things?".
Well, one way I use to explain this to non-programmers is to think about two CPUs, an old one with SSE2 and a recent one with AVX2.
Think about the SSE2 CPU as a "dumb child" who can only do addition and the AVX2 one as a "smart child" who can do multiplication.
Of course, as long as the teacher assigns those children additions, both are gonna be fine, however, if the teacher assigns those children multiplications, the second child (the smart one) will do them, while the first one (the dumb one) will have to split them into additions and then will get to the same result.
The teacher is like the program you are trying to execute.
Additions is like a program compiled with SSE2.
Multiplication is like a program compiled with AVX2.
If you try to assign multiplications to the dumb child without splitting them into additions, the dumb child won't be able to do them, which means that if you compile a program with AVX2 and you try to open it with an SSE2 capable CPU it won't be able to run it and will fail, however if you split them in sums and you assign them to the dumb child, he will be able to do them, but it will take longer (i.e if you compile the program with SSE2, the CPU will be able to run the program but it will be slower).

To that I'm gonna add a few things:

1) Different compilers make different assembly optimizations, therefore you see binaries compiled with Intel Parallel Studio (ICC), some others compiled with Visual Studio (MSVC), some other compiled with Clang and so on. In some cases Intel Parallel Studio produces faster code for the same instruction set target (like AVX2), some other times Clang does... It really depends, there isn't a "pick one to rule them all".

2) Sometimes targeting things like AVX512 might be useless if there's nothing inside your program that make use of those. Same for other instruction sets.

3) For those who don't like the very stupid example and wanna know how multiplications and divisions are done in real life, they're done with Shift Registers and have nothing to do with instructions set, but I like to use that example as it's like thinking about a CPU that doesn't have any shift registers (for which I made a video in 2020 during the pandemic https://www.youtube.com/watch?v=PGaBIR_ ... 8&index=46).

4) As a quick and dirty method if you don't wanna know any of this, just download Piriform Speccy, go to the "CPU" section and look at which instructions it supports.

[lazbert123]

That's a great explanation thanks! Had to go over it a few times but it all makes sense

I think what's happened is the CPU we've got in this server isn't that new, being an Intel Xeon E5-2650 v2 from about 2013 according to CPU-World:

Extensions & Technologies
MMX instructions
SSE / Streaming SIMD Extensions
SSE2 / Streaming SIMD Extensions 2
SSE3 / Streaming SIMD Extensions 3
SSSE3 / Supplemental Streaming SIMD Extensions 3
SSE4 / SSE4.1 + SSE4.2 / Streaming SIMD Extensions 4 ?
AES / Advanced Encryption Standard instructions
AVX / Advanced Vector Extensions
F16C / 16-bit Floating-Point conversion instructions
EM64T / Extended Memory 64 technology / Intel 64 ?
HT / Hyper-Threading technology ?
VT-x / Virtualization technology ?
VT-d / Virtualization for directed I/O
TBT 2.0 / Turbo Boost technology 2.0

https://www.cpu-world.com/CPUs/Xeon/Int ... %20v2.html

Looks like it has AVX but not AVX2 from that list. I'd like to get the server upgraded but alas that's not my call unfortunately... one day!

[emcodem]

Intel Xeon E5-2650 v2

thats an old friend of mine I guess about 50% of the servers i did take care about at work did or do still have this processor
Btw, are you sure that you need the resize module in combination with the A/V decoder at all? The encoding processors usually do resize on their own...

[FranceBB]

Intel Xeon E5-2650 v2

thats an old friend of mine I guess about 50% of the servers i did take care about at work did or do still have this processor

LOL that's really sad.
I mean, you have super fast 10 Gbit/s but an old 8c/16th AVX only Xeon
I have a slow old 1 Gbit/s only, but a shiny 56c/112th AVX-512 Xeon
If only we worked in the same company and we could combine the two... Life is unfair (T_T)