Hi there
My Holy grail would be creating a workflow that does the consolidate outside AVID so i have been playing around with the transwrap function, i can see that it is dependent on bmx created by BBC, so just to be certain before i spend more time, the transwrap function doesn't support every format/codec that AVID can consolidate, but AVID supports everything from bmx?
bmx trans wrap
Re: bmx trans wrap
Hey Da_K,
Anyway, the question as such dont make a lot of sense to me, usually all stuff that goes to edits should be transcoded in the ingest process. So in a normal workflow, you don't have a lot of different Formats/Codecs that the Edits have to deal with.
i guess it is a no in both directions. BMX is a tool dedicated to handle MXF and AVID is not really know to support "everything" in terms of Codecs/Formats.
Anyway, the question as such dont make a lot of sense to me, usually all stuff that goes to edits should be transcoded in the ingest process. So in a normal workflow, you don't have a lot of different Formats/Codecs that the Edits have to deal with.
emcodem, wrapping since 2009 you got the rhyme?
Re: bmx trans wrap
Hi Da_K,
by "consolidating" I assume you mean linking the asset from a bin and consolidate the asset in a workspace so that it's checked in inside Interplay, right?
If that's the case, then the answer is "it depends".
You would need to:
1) Create a workflow that takes everything in input through a watchfolder or an API call or whatever
2) Use a conditional that checks whether it's an .mxf or not
2.1) If no, then transcode to whatever mezzanine file you need
2.2) if yes, then use the command line executor to remux to .mxf using the BBC BMX Transwrap muxer
3) Check whether it failed or not
3.1) If the remux failed, then go back to 2.1
3.2) If it went through, deliver the final output
Alternatively, you can just add an encoding node targeting the format you want in output, set the muxer to MXF (BMX), set it to "Transwrap when possible" and call it a day.
Anyway, please note that the output is still not consolidated inside Interplay.
Such an output must be passed to AVID Mediadirector which will take the .mxf and check that one into Interplay for you in whatever workspace you want by consolidating the file without any additional processing.
Unfortunately, you need Mediadirector 'cause we don't have a node that is integrated with AVID Interplay and makes SOAP Calls, but if you're an interplay / Mediacentral user you should have it included already anyway in your bundle.
Last but not least, whenever it comes to AVID and Interplay, you're right, you should ALWAYS use BBC BMX Transwrap as a muxer.
The reason is that the FFMpeg mxf muxer is broken in so many ways that the odds of ending up with a non compliant file are very high and rest assured that at the very minimal "inconsistency", AVID will refuse it.
I've honestly lost track of how many times I reported issues related to it (where "it" stands for the FFMpeg mxf muxer) and at one point I just gave up and stopped reporting... https://trac.ffmpeg.org/ticket/9647 - https://trac.ffmpeg.org/ticket/10001 (and plenty more I reported which were left unfixed but I forgot the link)...
by "consolidating" I assume you mean linking the asset from a bin and consolidate the asset in a workspace so that it's checked in inside Interplay, right?
If that's the case, then the answer is "it depends".
You would need to:
1) Create a workflow that takes everything in input through a watchfolder or an API call or whatever
2) Use a conditional that checks whether it's an .mxf or not
2.1) If no, then transcode to whatever mezzanine file you need
2.2) if yes, then use the command line executor to remux to .mxf using the BBC BMX Transwrap muxer
3) Check whether it failed or not
3.1) If the remux failed, then go back to 2.1
3.2) If it went through, deliver the final output
Alternatively, you can just add an encoding node targeting the format you want in output, set the muxer to MXF (BMX), set it to "Transwrap when possible" and call it a day.
Anyway, please note that the output is still not consolidated inside Interplay.
Such an output must be passed to AVID Mediadirector which will take the .mxf and check that one into Interplay for you in whatever workspace you want by consolidating the file without any additional processing.
Unfortunately, you need Mediadirector 'cause we don't have a node that is integrated with AVID Interplay and makes SOAP Calls, but if you're an interplay / Mediacentral user you should have it included already anyway in your bundle.
Last but not least, whenever it comes to AVID and Interplay, you're right, you should ALWAYS use BBC BMX Transwrap as a muxer.
The reason is that the FFMpeg mxf muxer is broken in so many ways that the odds of ending up with a non compliant file are very high and rest assured that at the very minimal "inconsistency", AVID will refuse it.
I've honestly lost track of how many times I reported issues related to it (where "it" stands for the FFMpeg mxf muxer) and at one point I just gave up and stopped reporting... https://trac.ffmpeg.org/ticket/9647 - https://trac.ffmpeg.org/ticket/10001 (and plenty more I reported which were left unfixed but I forgot the link)...
Re: bmx trans wrap
Thank you for the detailed reply, in this case it was just an ordinary AVID workflow where we now go through the AVID std procedure of "ama" linking and then consolidating it onto a NEXIS. The thing is that AVID will handle most footage natively as long as it's in it's own MXF files. I don't want to Transcode.
If i was able to run my media through a FFA workflow and get AVID mediafiles it would be so much easier.
Ohh well
If i was able to run my media through a FFA workflow and get AVID mediafiles it would be so much easier.
Ohh well
Re: bmx trans wrap
Right, you can do something like this:
It will take an input, try to remux it to mxf via FFMpeg and then BBC BMX Transwrap and deliver the output.
This is because BBC BMX Transwrap only works with other mxf files (or discrete files like .wav and .h264 etc), but there might be for instance .mov files that contain codecs that can be muxed to .mxf like H.264 Intra and PCM audio, just to name a combination etc.
Anyway, if remuxing fails, it will try to re-encode to DNxHR using the same resolution, framerate, colormatrix, transfer and primaries and audio channels layout of the source and then deliver the file.
By the way, I gotta tell you that link and consolidate into Nexis can be done automatically and that's exactly what we're doing.
Let me show you a workflow called "Streamix".
This workflow receives files which people upload to Aspera.
Those files are downloaded using Aspera Orchestrator and then passed to FFAStrans through an REST-API call.
FFAStrans starts, does its bit of Avisynth magic with some logic and eventually creates an XDCAM-50 .mxf file.
Such a file is then passed to AVID Mediadirector which consolidates the asset to the Nexist storage (incoming media workspace) and checks that one into AVID Interplay.
and sure enough the file is now consolidated in a workspace:
It will take an input, try to remux it to mxf via FFMpeg and then BBC BMX Transwrap and deliver the output.
This is because BBC BMX Transwrap only works with other mxf files (or discrete files like .wav and .h264 etc), but there might be for instance .mov files that contain codecs that can be muxed to .mxf like H.264 Intra and PCM audio, just to name a combination etc.
Anyway, if remuxing fails, it will try to re-encode to DNxHR using the same resolution, framerate, colormatrix, transfer and primaries and audio channels layout of the source and then deliver the file.
By the way, I gotta tell you that link and consolidate into Nexis can be done automatically and that's exactly what we're doing.
Let me show you a workflow called "Streamix".
This workflow receives files which people upload to Aspera.
Those files are downloaded using Aspera Orchestrator and then passed to FFAStrans through an REST-API call.
FFAStrans starts, does its bit of Avisynth magic with some logic and eventually creates an XDCAM-50 .mxf file.
Such a file is then passed to AVID Mediadirector which consolidates the asset to the Nexist storage (incoming media workspace) and checks that one into AVID Interplay.
- Screenshot from 2023-04-07 08-47-52.png (55.45 KiB) Viewed 3963 times
- Screenshot from 2023-04-07 08-49-16.png (23.94 KiB) Viewed 3963 times
- Screenshot from 2023-04-07 08-53-32.png (89.16 KiB) Viewed 3963 times
- Screenshot from 2023-04-07 08-54-21.png (345.44 KiB) Viewed 3963 times
Re: bmx trans wrap
In addition to what Frank said, you can always use raw2bmx instead of bmxtranswrap. This way at least you eliminate all wrapper related issues that bmxtranswrap cannot read (e.g. mov to mxf).
You can see the list of supported videos and audios when calling raw2bmx (it comes with ffastrans too) without parameters in the "Inputs:" Section.
Anyway what you want to do sounds like really a huge amount of work and a little bit "dangerous" too because you might produce something that looks compatible in first place but while you edit with it, avid crashes because of some incompatibility mid-stream...
You can see the list of supported videos and audios when calling raw2bmx (it comes with ffastrans too) without parameters in the "Inputs:" Section.
Anyway what you want to do sounds like really a huge amount of work and a little bit "dangerous" too because you might produce something that looks compatible in first place but while you edit with it, avid crashes because of some incompatibility mid-stream...
emcodem, wrapping since 2009 you got the rhyme?
Re: bmx trans wrap
Indeed! AVID is extremely picky about formats.
For instance, AVC Ultra from Panasonic is, by definition, VBR, but it has some very strict limits in the thresholds.
When we were creating the AVC Ultra encoding node in FFAStrans in summer 2021, I remember making some experiments / tests out of curiosity, so although I set the right --bitrate and --vbv-maxrate I raised the --vbv-buffersize a bit.
Needless to say, AVID accepted the file just fine, consolidated it and checked that one into the system, but upon playback when it reached a certain scene it crashed with a buffer overflow.
Nowadays the AVC Ultra is of course with the right parameters and plays just fine (not yet available in the public build, sorry folks), but this is just to reiterate exactly what Emcodem said: a file might LOOK ok and be consolidated, but not be ok at all and fail upon playback.
This is particularly dangerous if you're doing hotroll from a Media Composer to Playout via SDI, for instance.
Anyway, in a nutshell, unless you have a very good reason, always re-encode to a safe format and then check that one in.
Re: bmx trans wrap
That is a very nice workflow, i don't have an Interplay so i have to use some other tricks to get my stuff from FFA online
As a principle, we always keeps our recordings as close to the source quality as possible, so we very rarely re-encode anything. That is why i was asking about the MXF rewrap
Since AVID really just does a MXF rewrap on supported codecs/formats when you link and consolidate it would seem logical that the same function could be achieved outside AVID. But i guess that would require a reverse engineering job on MXF files created by AVID for every format AVID supports.
Re: bmx trans wrap
Before i forget, of course instead of using raw2bmx which would most likely deliver the best result in terms of compatibility, you can potentially just use ffmpeg for a "pre-mux" into op1a and feed the ffmpeg mxf into bmxtranswrap. It is actually what ffastrans usually does when you set the output wrapper to bmx.
Example, ffmpeg reads test.mxf with codec copy, produces an op1a and pipes it live to bmx, which produces the avid opatom files.
But piping is an advanced technique. I'd recommend for you to first get everything working by first letting ffmpeg write the op1a file and then remuxing with bmxtranswrap. If that works for your input formats, you can go on and pipe the commands in order to save time and I/O's.
Example, ffmpeg reads test.mxf with codec copy, produces an op1a and pipes it live to bmx, which produces the avid opatom files.
But piping is an advanced technique. I'd recommend for you to first get everything working by first letting ffmpeg write the op1a file and then remuxing with bmxtranswrap. If that works for your input formats, you can go on and pipe the commands in order to save time and I/O's.
Code: Select all
C:\FFAStrans1.3.1\Processors\ffmpeg\x64\ffmpeg.exe -i "C:\temp\test.mxf" -codec copy -map 0:v -map 0:a -f mxf - | "C:\FFAStrans1.3.1\Processors\mxf_tools\bmxtranswrap.exe" --log-level 0 -t avid --project "EXAMPLE" --clip "EXAMCLIP" --tape "EXAMPTAPE" -o "c:\temp\avidstuff" -emcodem, wrapping since 2009 you got the rhyme?