iPhone spatial audio issue

[crispyjones]

Recently ran into an issue where videos from the latest iPhones were failing in both my Vantage(flip64) and FFAStrans worfklows. After poking around I find that their spatial audio, tagged as "apac", shows up as a second four channel audio stream. Both Vantage and FFAStrans default is to decode all audio streams so the user can do what they want with them downstream, unfortunately this fails when one of those audio streams has no decoders available. It's unfortunate as the first stream is completely normal AAC stereo stream.

I've attached(Merry Xmas!) my current workaround in FFAStrans, a snip from a larger workflow. It uses mediainfo to see if apac is present, if yes then it hits a custom FFMPEG block that removes the second audio stream. It's clunky and will break if any of the assumptions change, so if the codec name changes from apac, or if it's not the second audio stream, etc.

This brings up a feature request for the devs. Perhaps when no decoder is available instead of failing the whole thing just ignore the non-decodable stream? Or maybe there is a better way to do this with existing workflow nodes?

[emcodem]

Aye Mr. Crispy, i am excited to see you back in business Thank you so much for your contribution, very much appreciated!

It would be good to have some sample file, unfortunately i did not find any. I'm pretty sure steinar wants to add support for it and if not then i shall implement it.
There are also chances that we get a fix for the failure because ffmpeg has an apac decoder. If we can submit some sample file in the ffmpeg bugtracker, there are chances that someone takes care about identifying and fixing the shortcoming.

If by chance anyone has such a phone and can produce a file for debugging, please generate the shortest and smallest possible file that you can. E.g. just film some clap for 0.5 seconds, lowest possible resolution and framerate, hevc encoding. The use of this can potentially be to end up in the ffmpeg "fate" suite which would ensure that this exact file type stays supported in future.

[FranceBB]

Vantage? Brrrrrrr that word is like Voldemort in Harry Potter, it can't be pronounced here.
Besides, it should really be called "DisadVantage" given that it sets you back compared to using FFAStrans.
FFAStrans for life!
Ok, jokes aside, I'm very happy to see that you reported this 'cause this is perfect for testing! I actually faced the same issue and so did Grandmaster Steinar who implemented a workaround in the master branch two weeks ago. (Yes, he's fast). As of now, the latest FFAStrans 1.4.1 should be able to handle this.

You can actually test it yourself if you click on "code" and "download .zip".

https://github.com/steipal/FFAStrans-Public

This is why I'm happy to see that you reported it: we actually need more people to test it.
When you go back to the office please try it out 'cause the tests we can make on our own will always be limited and we're always happy to have other people try things out. This is especially important if you have different recordings with different devices.

By the way, the fix is implemented in exactly the same way you described, namely in the audio functions not only FFAStrans checks that the codec type is audio using ffprobe like before but now it also makes sure that the codec is known by checking the codec name. If it's not known, then it doesn't attempt to decode it, thus solving the issue. This should be future proof (which is what you would expect from Grandmaster) as it's not limited to this particular scenario but also any other possible future codec until an open source decoder is available.
Anyway, please let us know and remember that FFAStrans 1.4.1 was built directly from master (well... fine, before the other guys scold me saying that it wasn't, it was built from the 1.4.1 branch which right now is master anyway ) so there might be other bugs as it hasn't been released as stable just yet.

I'm eager to know the results but if it works then happy days!

Oh, I almost forgot: Merry Christmas!

[crispyjones]

I can confirm that the very latest version of the master branch downloaded as @FranceBB directed seems to deal with apac/spatial audio without failing. It's only installed on my desktop that I use for testing, haven't tried it in production. I can also confirm that the latest version downloaded via the releases page (1.4.0.7) will fail on the same file. So it does appear @admin has figured this out since that release.

@emcodem I am working to get you a test file that is 1) tiny enough to upload here and 2) I feel comfortable uploading as the other files are not mine to distribute.

I must stick up for Vantage, I've dealt with companies that will sell you far less, have worse support, all for much more money$$$. I've been using their product since it was called "Flip Factory" back about 2011ish. Their killer-app for our operation is Avid (the real Voldemort) integration. Their GPU accelerated "lightspeed" is also very faaaaasssst! And (hate to admit it) I'd rather tell my boss "the thing corporate bought us broke and it's not my fault they can't fix it" vs "the thing I downloaded that I'm the only who can work on broke and I haven't figure it out yet....I guess it is my fault?"

Seriously though, if I didn't need the Avid stuff, FFAStrans could do everything we need and is genuinely amazing. FFAStrans for life!

[FranceBB]

I can confirm that the very latest version of the master branch seems to deal with apac/spatial audio without failing.

Yaaaaay, music to my ears!
In your face, Telestr- ehm I mean, cool cool, I'm glad to hear that it works.

@emcodem I am working to get you a test file that is 1) tiny enough to upload here and 2) I feel comfortable uploading as the other files are not mine to distribute.

Thank you for this. With that we can then go to the creators of the various indexers (the likes of libav) and the rest of the FFMpeg community and allow them to create a proper decoder for it.

Their killer-app for our operation is Avid (the real Voldemort) integration.

well I also have AVID but I'm currently using AVID Mediadirector to perform the Interplay check-in. Anyway, story for another day.

[crispyjones]

Here is a very short clap video shot with spatial audio/apac on an iPhone 16. I've confirmed it fails in 1.4.0.7 and older, but works in the version that is 20+ commits ahead I downloaded from GitHub on 12/26. I believe this file is was also shot in spatial video mode, I'll try and get a cleaner apac only file.

clap.MOV

iPhone 16 encoded with spatial audio

(1.97 MiB) Downloaded 28 times

[emcodem]

Fantastic, thank you so much for it! Also as it's an eye catcher, i love the pants, where can i get these
Just to be sure, you are ok with us to share the stuff in ffmpeg bug tracker which makes it even more public than it already is?
I believe the file itself should be fine as it is, let's see if any ffmpeg dev asks for something different.

Cheers!
Sidenote, i think flipfactory was alread buried in 2011 hehe
Also, the problem of "only you can operate ffastrans" might also apply to many other tools, if not even vantage itself. If you want others to be able to operate it you can just train them. Depending on the size of the organisation, the topic might be interesiting for multiple departments so you can potentially train multiple people at once, that might even be more efficient than the usual concept with vendors tools which are bought for special purposes like vantage and co.

[crispyjones]

Free to use!

Flipfactory was already buried in 2011? Impossible...that means I'm way older than I think I am!

[emcodem]

Impossible...that means I'm way older than I think I am!

Thank you so much and happy new year... lets see where this goes.