Hi
This topic must find its way into the Wiki but it will require a lot of resources because it most be done very scientific and 100% reproduceable.
I do use ffastrans on a lot different configurations, depending on the usecase (which workflow?) and available
hardware. GPU is for consumer and web (proxy) stuff only but i mostly create Mezzanine formats (for production) like XDCAMHD or XAVC Class 480, sometimes even Prores in various configurations.
In general i can say that virtual machines only make sense for very special setups as my transcoding servers typically use all available CPU's most of the day (except for high quality filtering workflows), which would not really make sense for a virtual machine. Also all the
Hardware Piping to virtual machines for using Intel, Nvidia or other
Hardware encoders is not easy to get on a virtual machine.
veks wrote: ↑Mon Mar 02, 2020 3:44 pm
It would be great to see FFAStrans support NVIDIA CUDA encoding
Ok, up in front, ffastrans does not yet have much focus on the "delivery" sector but what you want basically only serves the delivery sector (instead of the production sector). Even if we would start working on the delivery sector, we would probably first go for HLS and multibitrate because this topic is much more important than supporting those
Hardware encoders that are built into Graphics cards.
Personally i would never use CUDA at all for encoding purposes (only for filtering). If i go for
Hardware Encoding, i typically use Quicksync or NVenc.
We had some testing session in that direction and it did not look like we can easily integrate the currently existing ffmpeg encoders for quicksync and nvenc. The reason is that ffmpeg does not yet have it completely integrated and we dont want to workaround and reverse engineer all the flaws. However, as it can be used for specific workflows (e.g. always the same input file format) anyway, you can easily integrate it with a custom ffmpeg node.
veks wrote: ↑Mon Mar 02, 2020 3:44 pm
As GPU encoding is far faster and cheaper compared to CPU
Sorry but that sounds like you jumped on the train of rumours and advertising. In general you are totally wrong. In detail you can achieve very special workflows using Graphics boards a few dollars cheaper than using CPU because of the Power cost but if you calculate the needed engineering for it you will see that you burnt many thousand dollars for it - or maybe even hundreds of thousands, so it only pays off in extemely high scale (like thousands of servers)
OK, first typically you dont use the GPU for encoding. The graphics boards from INTEL, NVIDIA and AMD have special chips onboard called ASIC (Application Specific Integrated Circuit) which allow to encode into a limited set of codecs and presets. The GPU is not even really used when using those ASICS. But there is other stuff like filtering that we developers can use the GPU (e.g. using CUDA on NVIDIA cards)
Anyway, my tests show that you can typically do about 3 parallell FullHD encodings in low quality on one Nvidia GTX 1080, each will have about 2.5 realtime while on a CPU like Core I7 9700K you can do 11x realtime encoding without any tuning.
Also, the ASICs always have different limitations like no high bitrate encoding and no 4:2:2 support which makes them only useable for consumer stuff like gaming industry (live streaming to twitch what you currently play - thats what it is built for). No production codecs can be done currently using that
Hardware. Matrox and i think Rhode&Schwarz has some ASICS for Production codecs, their cards start from 7k€
veks wrote: ↑Mon Mar 02, 2020 3:44 pm
P.S. do you have some benchmarks for these servers, how much simultaneous transcoding you can do depending on source and transcoded profiles? And other information related to it.
This topic is not easy at all, it mostly depends on the input format and conversions that you do. I can benchmark some CPU's and ASIC's for you if you tell me what exactly you want to go for, e.g. speed for one single encoding at lowest possible quality or highest quality or such....
Ghtais wrote: ↑Mon Mar 02, 2020 4:17 pm
64Go ram is unneeded as FFAStrans doesn't use more than 10 GO.
Unfortunately it doesn't take all CPU power during transcoding. I don't know why. It depends of your workflow, with H264 processor it takes only 16% CPU power.
FFAStrans is just driving a workflow, it does not use lots of RAM at all, typicall just about a few MB of RAM. FFmpeg and Avisynth uses most RAM and their demands depend on a lot of factors.
The RAM comes into use when dealing with UHD material. The companies i work for typically buy all the servers with minimum 128GB because the price of the RAM does not really matter anymore and it is always better to have more than less
H.264 encoding alone should typically take most of your CPU, independent of the settings. 16% sounds like it uses exactly ONE physical Core which might most likely relate to some filtering.
In the End, for production purposes, it all comes down to Using a Core I processor instead of a XEON if you want to go fast for a single stream or do filtering, typically because of the higher CPU clock.
