DNxHD encoding issues

[Vininski]

So that dud clip I uploaded is importing now, though I've tried multiple times to re-import clips that failed me earlier. I'm thinking now this problem may lie solely with Avid or at least the install of Avid on the machine I'm using. I just tried re-importing 4 dud files I'd kept a copy of and the first time 1 showed as offline, another spat an error and the other two were perfectly fine. I deleted the .mdb + .pmr and tried again as I have done in the past and suddenly all 4 are working fine...

I think I'm going to have to re-install Avid and if that doesn't fix, start a new project and import the bins to. The woes of Avid...

Thanks to emcodem, Benjamin and steinar for all your help! I'll have to check out those tools and steps to if I run into more problematic files.

EDIT: Just saw your edit Benjamin, I was thinking of actually doing a screen recording to show exactly what is happening as I am probably not explaining it very well. I'm at the end of my shift now but I'll record the next batch that arrives tomorrow.

[momocampo]

Hi,

Ok no problem
I can tell since few years Avid database aren't so reliable than before. I experimented some issues about these database.
Anyway, before create these database, you can without problem play the video atom file with vlc. Beware, Vlc reads almost all files but it's not a check.
Last thing, I don't know how you work but maybe you can work with AMA Link instead of transcode all your files, it can be a quickly workflow.
Cheers.

Benjamin

[Vininski]

Interestingly VLC no longer plays these files or the files from camera on this machine, a recent issue that I haven't had the time to look into yet. One thing at a time! I used it a lot to so it's been a nuisance.

We've worked a bit with AMA linking before and I quickly ruled against it as it cause a large number of issues and was generally sluggish in Avid. I'm working on large projects with between 1-88 shoot days and multiple cameras so the number of files adds up fast and the performance hit is noticeable.

I've linked a screen recording of previously mentioned weird behaviour. This time the clip is referencing the previous clip (101) instead of itself and it's respective mediafiles (102) explaining why it doesn't show on first import but does if brought into a new bin.

(Skip to around 00:2:50:00, it's mostly loading)
https://drive.google.com/open?id=1Fwx6Q ... a6U7BF5C3y

[momocampo]

Hi,

Well, it seems quite difficult to see why is the problem because I don't know the beginning of your production. From where come all these files? How many files had you before create mxf Atom files? Are your atom files have the same duration than the originals? etc.
The first thing you can check it's if the number of originals files are exactly the same than the Atom files. Then, check the duration into avid. You will see if the ffastrans conversion is ok.
The TC issue is maybe a fps issue. 25fps into 29 ? Or something else?
Anyway, check if you have the same number of files before and after conversion and we will see.
Cheers.

Benjamin

PS: You're right about Link, not very reliable and really less than Avid Mediafiles folder...

[Vininski]

Those atom files are straight from ffatrans (input and output all on the same drive so it's quick to move into the Avid Mediafiles folder). It creates the correct number of files, I've checked every time and there are always files for the clips that are "missing". I haven't checked the duration though, I'll have a look at that next.

[momocampo]

Ok, I asked the duration because I wonder if it can be a encoding error. Sometimes, a corrupted file is encoded but the file isn't complete, the file is encoded until the error and directly delivered. This kind of error could be "brake" file metadata and Avid could be lost.
I must watch again your video file to understand.
Cheers.

Benjamin

[emcodem]

Hey
we already know that the clips are fine from a structural perspective, so experiments in direction of duration etc... seem useless to me.

@Vininski: i struggle a little with your screenshot not showing whats wrong with the clips you already uploaded... typically when dealing with such problems, i concentrate on using always the same "works" and "don't work" samples.
However, from your video, i see that your problem is that the clip FRS3_SD07_A008_Clip0102 seems to reference FRS3_SD07_A008_Clip0101.

Can you confirm that mediainfo shows the correct package name for clip FRS3_SD07_A008_Clip0102?

Here:

packagename.png (37.11 KiB) Viewed 10196 times

[Vininski]

Sorry about that, I should have kept the dud files for troubleshooting. I'll keep aside a copy of the next file that causes issues and record that. It's been a hectic week in the office, I'm barely keeping up!

[Vininski]

I've only just had the time to do another test and record the issue clearly (I've been having the issue all this time). I've uploaded the 3 clips and video recordings of the issue. Here's what happens:

"2019-06-20_all-clips"
Copy transcoded OP-Atom files (3 clips, numbered 2,3 and 4) from FFAstrans output to new folder in Avid Mediafiles/MXF
Let Avid scan and generate MDB file
Drag MDB file into new avid bin, only clips 2 and 4 appear
Create new folder in Avid Mediafiles/MXF
Cut OP-Atom files of missing clip 3 and paste into this new folder
Let Avid scan and generate MDB file
Drag MDB file into new avid bin, clip 2 appears
Check reference files of clip 2 in both bins to confirm that they reference the clip 2 OP-Atom files

"2019-06-20_just-clip-3"
Move OP-Atom files for clips 2 and 4 out of Mediafiles/MXF (Avid will no longer be able to access these files, they would appear "Media offline")
Delete old bins and MDB file for clip 3 OP-Atom files
Drag MDB file into new avid bin, clip 3 appears
Check reference files of clip 3 to confirm that they reference the clip 3 OP-Atom files.

This explains why the files I uploaded earlier appeared to be fine, the issue seems to only occur when importing multiple files. So far it's always been files in sequence. (eg clip 2 will reference clip 1 and clip 55 will reference clip 54) I'm still not sure if this issue has anything to do with FFAstrans but hopefully someone can help me figure it out.

Link: https://drive.google.com/open?id=1IdeDy ... oIJW9JC9fX

EDIT: Forgot to add that yes all these clips and the previous example showed the correct "Package name" in mediainfo. Still have no idea why they are not being referenced correctly.

[emcodem]

Hey Vinny,

sorry for the delay.
OK, i found out some very disturbing thing about your 3 files, (perfect examples and screenrecordings, thanks!): they all have the same "file_package_umid", here how the umid looks like, you can use ffprobe to see it.

umid.png (30.62 KiB) Viewed 10007 times

Now as i understand, AVID relies on those umids so it might be the cause of your issue. I did some testing how you can influence the UMID and it looks like it is generated automatically out of the project name and or tape name you enter in the dnhxh encoding node. It does not seem to consider the "clip name" which indicates that the guys from BBC that developed bmxtranswrap assumed that there will be only one mxf file per tape and it will contain all clips of the tape. (I am not even sure if this is possible in an ATOM file)

So i guess you can solve your issue by making sure to have a unique tape name (add %s_original_name% to the tape name or so).
In case this turns out to work and you desperately need the tape name to be exactly the same for all files, you'd need to open a feature request in the forum and ask steinar to utilize the --mp-uid option with bmxtranswrap to create a unique MP for every file that was created. I am not sure if having a MP UID that is not created out of tape and reel name does conform to the standard but we would probably handle that topic in the corresponding thread then.

Could you please test this?

cheers,