i hate pytorch

[taner]

Hi Admin Team,

Look at this: https://github.com/chaiNNer-org/chaiNNer
Node based image manipulation gui.
I'm currently using it to compare different super resolution models against each other so I can implement my favorites in FFAStrans in the end.
Because there are trillions of models and I was looking for a more easy way to handle the comparison.
What is great about Chainner is that it can handle those models natively (sure you have to import it first).
But then all combinations and filters are easy available and usable.
You know what would be great?
If FFAStrans could do it too

Apart from that it has some nice features which may be an inspiration for FFAStrans development.

Best
Taner

[emcodem]

Well you need a cli tool for automation. Does this work for you? https://github.com/chaiNNer-org/chaiNNe ... ocs/cli.md
Also i am sure @FranceBB will tell you why he doesnt like the GAN models

[taner]

hehehe yeah I'm not a big fan of the gan models too.
But it really depends what the expectations are.
For myself I want a slight enhancement of medium distorted images.
No artificial crap outcome where faces and objects have no texture anymore and identity changes of faces.
And I think with some combinations it could be achieved, at least for some scenarios.
I mean...FFAStrans is great as I then can provide a simple watchfolder solution for the redaktion und editoren which would definitely save some time for them.

[FranceBB]

I'm gonna quote your topic.
Basically the "I hate pytorch" part.
I mean, I took a look at the GitHub you linked and they're not using anything new, they're just leveraging on the different ESRGAN models, therefore why would you wanna make your life miserable with Python when we have a C++ interface for them that can be used in Avisynth and is already supported by FFAStrans out of the box?

https://forum.doom9.org/showthread.php?t=184768

All you need is to either pick an ONNX model OR convert other models to .onnx and you're gonna be ready to go.

Just download the mlrt plugin from here: https://github.com/Asd-g/avs-mlrt/releases
Download the models from here: https://www.dropbox.com/sh/f74ao9t1qarf ... r6M4a?dl=0

Create a FFAStrans workflow with the A/V Decoder and a Custom Avisynth Script like:

Code: Select all

#Path to the dll
LoadPlugin("\\FFAStrans\Processors\avs_plugins\mlrt_ncnn.dll")

#Bring everything to 32bit float
ConvertBits(m_clip, 32)
ConvertToPlanarRGB()

#Use the GAN model with 32bit float precision
mlrt_ncnn(network_path="\\avs000\Ingest\MEDIA\temp\onnx-models\realesr-general-wdn-x4v3_opset16.onnx")

#Dithering down to the output bitdepth
m_clip=ConvertBits(bits=8, dither=1)

return m_clip

and then add an encoding node to encode to whatever output you want.

A couple of notes:

1) Please note that GAN models only work with progressive clips, so you may wanna add a deinterlacer after the A/V Decoder just in case
2) The dithering to output bit depth has to be adjusted to your needs according to what the output is gonna be. Also please note that you MUST put ConvertBits() there 'cause even though Avisynth can work in 32bit float with as many values as you'll ever need, FFMpeg can't, which is why you can either deliver 16bit planar (which FFMpeg understands) or go directly down to whatever bit depth you want in output.
3) GAN models will be processed through mlrt_ncnn() but require a GPU and a LOT of VRAM. Keep in mind that my poor people GTX 980Ti couldn't render anything when I tested them at home and I had to go to the office to "borrow" some Quadro cards. That aside, nowadays consumer cards are just fine, so if you have a 4090 etc you're gonna be fine too, but I'm poor, so...

Also i am sure @FranceBB will tell you why he doesnt like the GAN models

Rather than going through several lines on why non deterministic upscales cannot be reliably used for archival purposes, I'm gonna let the screenshots talk:

Code: Select all

#Indexing
LWLibavVideoSource("Test.mxf")
#ImageSource("\\mibctvan000.avid.mi.bc.sky.it\Ingest\MEDIA\temp\Lenna_(test_image).png")
Bob().Spline64Resize(848, 480)
original=ConvertBits(8).Converttoyv12().Text("Original", y=66)

#Downscale
SinPowResizeMT(width/4, height/4)

#Various Upscales
point=PointResize(width*4, height*4).ConvertBits(8).Converttoyv12()
#bilinear=BilinearResize(width*4, height*4).Converttoyv12()
nnedi3=nnedi3_rpow2(cshift="Spline64ResizeMT", rfactor=2, fwidth=width*4, fheight=height*4, nsize=4, nns=4, qual=1, etype=0, pscrn=2, threads=0, csresize=true, mpeg2=true, threads_rs=0, logicalCores_rs=true, MaxPhysCore_rs=true, SetAffinity_rs=false).ConvertBits(8).Converttoyv12()
esrgan=last.ConverttoPlanarRGB().ConvertBits(32).mlrt_ncnn(network_path="\\myshare\Ingest\MEDIA\temp\realesr-general-wdn-x4v3_opset16.onnx", builtin=false, list_gpu=false, fp16=true).ConvertBits(8).Converttoyv12()

#SSIM
pnt=SSIM(original, point, "\\myshare\Ingest\MEDIA\temp\point3SSIM.csv", "\\myshare\Ingest\MEDIA\temp\point3SSIM.txt", lumimask=1, scaled=0).Text("PointResize", y=66)
nne=SSIM(original, nnedi3, "\\myshare\Ingest\MEDIA\temp\nnedi3SSIM.csv", "\\myshare\Ingest\MEDIA\temp\nnedi3SSIM.txt", lumimask=1, scaled=0).Text("NNEDI3", y=66)
esr=SSIM(original, esrgan, "\\myshare\Ingest\MEDIA\temp\esrganSSIM.csv", "\\myshare\Ingest\MEDIA\temp\esrganSSIM.txt", lumimask=1, scaled=0).Text("ESRGAN", y=66)

#Preview
a=StackHorizontal(original, pnt)
b=StackHorizontal(nne, esr)

StackVertical(a,b)

[emcodem]

Question is what happens when you apply the GAN stuff after the NNEDI scaling Or a chain of nnedi's?

[FranceBB]

You mean to use it only for its filtering ability rather than upscale abilities?
Well, simply enough, it filters the heck out of an image.
I gotta be fair, when it works, it does it very beautifully and better than traditional degrain and denoise algorithms like the MVTools based MDegrain3() or the FFTW based DFTTest(), however, unlike those very standard algorithms, it can also fail miserably.

Another example, just for the sake of.
If you take a look at the following screenshot, for instance, the image is crystal clear, but it removed the gates (which, of course, is not acceptable):



few more examples:

This one looks "canvas on oil" rather than a BetaCAM video (the ESRGAN one):




and, to some extents, so does this:



It is sharp, though, I'll give you that...

[taner]

Thanks a lot for your suggestions and the insight Frances!!!