How (and why) i use FFAStrans on MacOS (and linux etc..) and whisper

[emcodem]

We found out that a M1 Ultra Mac would deliver the same throughput while running speech-to-text (whisper) subtitling as a PC with A5000 Nvidia.
The reason is mainly because the main developer of whispercpp (github) is a Mac guy and focussed a lot on that. He built in special hardware support for the AI cores in the M1 (NEON framework i think). However, overall we are able to generate the same amount of subtitles using only ~25% of effective power which means that after 2-3 years operation the Mac saved it's own costs just by consuming less power and generating less heat.
Another very good point is that we can finally use the better whisper version because whispercpp is under heavy active development while the Const-me (GPU) version that we need for running on the nvidias does not really look like it's evolving at all.

Anyway, so even if i hate Macs from the bottom of my heart (not the linux part about them, just the mac part), i needed to find a way to integrate the Mac into an existing FFAStrans installation and distribute load between the nvidia on PC (which runs the ffastrans base installation) and the Mac.

If you read older topics in the forum, you also know that FFAStrans is pretty far away from being able to run natively on Linux due to the Scripting Language it consists of. So there is no way to actually "install ffastrans" on Linux (i was not able to get it running in wine too).

Anyway, so my way to do it was to develop a Plugin Processor that executes cmd commands via HTTP on a remote server. As i did not want the need to actually let Linux (macos) access the /db directory of ffastrans, of course there are some limitations but they might be less than you think.

The new "Remote cmd Executor" Processor is able to:
*) queue jobs totally independent of the ffastrans queue, based on a freely defined queue name and concurrency (this part is hard to explain actually)
*) submit jobs on any receiver that runs the job receiver application (comes pre-built for mac linux and windows with the plugin processor)
*) of course wait for jobs to finish
*) cancel jobs just as it was a normal ffastrans job, it will also send a cancel command to the remotely running commandline on linux device
*) report errrors and results to the workflow just as usual

Anyone interested, you find the processor where all plugin procs are:
https://www.ffastrans.com/wiki/doku.php ... processors

Also, i made a youtube video about it:
https://www.youtube.com/watch?v=vNtzGhRzWxc

[ThomasM]

How cool is that?

Great, so we can Re-use our Win10 Workstations (not win11 compatible...) with Linux!

Thank you, emcodem!

yippieyeay....

cheers,
Thomas

[taner]

Anyway, so even if i hate Macs from the bottom of my heart (not the linux part about them, just the mac part),

Hehehehehe

[emcodem]

Hehe @taner always picks up the important part of my messages
and @ThomasM thats a really neat idea, i'll probably pick up this concept in larger scale as soon as win10 fades out of support...

[andrezagato]

I'm not sure I understand all it could do. But I find it really interesting.
Would it be possible to use this to encode Prores files ? I know there already is a Prores node, but I'm not sure how reliable it would be to use it to deliver to big streaming platforms.

Great job emcodem.
Another thing. The video is a great tool to teach how to use FFastrans features, you guys should post more videos!

[emcodem]

I'm not sure I understand all it could do.

It is relatively easy, just forget about ffastrans for a while and think about what you can do using a shell command on your favourite Mac/linux/whatever host. All you need to do in your ffastrans workflow is to calculate your shell command and use remote executor node to execute it on your favourite external host.

If you want to encode prores using some native Apple encoders, again you first have to work out some shell command that does that. I don't really see a reason for it but i believe you could use an apple native encoder "prores_videotoolbox" codec in ffmpeg on the mac.
The problem with encoding is that usually shell commands needs lots of engineering around the parameters. On a native ffastrans windows host, using a native ffastrans encoder processor, ffastrans will do a lot of magic to calculate the very complex ffmpeg commandline that's needed to come from the current source file to the output (add empty audios, insert lots of filters etc). But ffastrans does not yet allow "access" to the internally calculated ffmpeg commandlines, so all the Encoding processors are limited to be executed on windows only. In future versions this might change.

If you have a usecase where the input is always kind of the same format so the shell command does not need lots of smart calculations, your workflow using this new Remote cmd executor would look like:
*) ffastrans windows host finds new file on NAS in watchfolder (just usual ffastrans workflow)
*) workflow translates %s_source% from windows to linux perspective (replaces \\server\share by /mnt/server/share) - assumes that you mounted your NAS share on the linux host
*) workflow can optionally do any other calculation you need to get your shell command constructed correctly
*) workflow executes the remote shell command and waits for it to finish
*) workflow translates the target filename back from linux to windows perspective and sets %s_source% to the new filename
*) workflow goes on on windows as usual

you guys should post more videos!

I thought you do that for us

[ThomasM]

This project by @emcodem could also become a "FFAStrans farming on steroids". Just thinking a little further.

you guys should post more videos!

I thought you do that for us

yeah...

And maybe we start a Thread linking to all the videos...

[FranceBB]

i'll probably pick up this concept in larger scale as soon as win10 fades out of support

The time will unfortunately come sooner than we might think... 2025...




(sorry for the poorly made aliased meme, but I had to make it)

Anyway this is really a shame 'cause Windows 10 has become really stable over the last 8 years while Windows 11 is still a picky, unstable, XAML mess and what's worse is that by 2025 we'll have to deal with Windows 12 (not yet announced by Microsoft, but there are early leaked documents around from Intel making their new CPU lineup, the ones that will drop the "i" in the name, Windows 12 compatible).
What's even worse is that although thus far Windows Server preserved some sanity (so much so that Windows Server 2022 still makes use of the old Server 2019 based UI and functionalities but with the Windows 11 optimizations and features), Jeffrey Snover (the creator of PowerShell and the one who has been responsible for every Windows Server version released so far since Server 2003) left Microsoft, so there's no guarantee that newer Server versions will be good either.
Microsoft what have you become...

[emcodem]

Nice image

The time will unfortunately come sooner than we might think... 2025..

My god, i was not aware about that it's so close...
It is kind of already too late for my company (again) to start the migration... Lots of new Hardware needs to be bought and lots of software vendors need to be kicked in the ass to test and support their stuff on Win 11 ^^ Thousands of PC Clients and Laptops need to be updated or replaced...
I'm not even sure if the last migration from Win 7 to 10 is finished yet. It's (again) a nightmare