FFAstrans : ingest and re import in AVID media composer

[Ghtais]

Hi all

I would like to share with you some very strange things that arrive to me the last few days. Perhaps someone can help me to understand what happen.
I usually ingest my footage with FFAStrans (with the XDcam or DNxHD node / Avid project structure enable). When the transcode is finished, I open media composer, waiting for scan, and drag and drop the .Mdb file in a bin to have my clip. It works great without any issues.
But, serious problems arrives when I want to re import some media that has disappeared for example when a stupid man hit "suppr" without thinking or when I crash my NEXIS, or when I want to conform proxy to HD resolution for example.
Ok so, I transcode a second time all my files with the same workflow, same name, etc. Avid scan my Avid mediafiles folder and I relink my clip with new medias.
Just after AVID has finished the scan, strange thinks appears on many timelines in my AVId project :
in my Avid project, I have some clip swapped with other one on some timelines. I mean they have same timecode, same length, same place but they coming from other Avid project. This new clips randomly replace one that has been placed before by the editor in the timeline giving a totally crazy editing with no sens. This new clips always coming from previous or last transcode with FFAStrans (never from ingest with AVID)

So I don't understand how it is possible. I was thing about a bug in BMX or something that deal with unique ID in the file.
In the XDCam node I check the "Create AVID project structure" like this:


Is this correct ? I don't really understand in what "project name" is needed in my case ?

if someone can explain what is happen or have the same issue and know how to deal with, it will be very helpful.
It is a very big deal because you can ruin one month of hard work in only 5 minutes (one scan).

thank you.

[emcodem]

Hey Ghatis,

what a huge unfortune, i am so sorry for you! Hope you were able to repair the issue by somehow relinking or so?
We worked on the UID Topic already and the owners updated BMX accordingly... is there any Chance that you can test the new build?
viewtopic.php?f=4&t=887&hilit=bmx&start=20#p3990

[momocampo]

Hello Encodem,
First, have you others media in your AvidMediafiles folder? If yes, it's the cause of your problem I think. When you use the process (ingest->mdb) ALL the medias in Avidmediafiles folder come. If you re create media with FFastrans again, create another folder in the Avidmediafiles folder called "2" (first called "1") then choose "2" folder as FFastrans destination folder. Your new media files will not be mix with others.
Perhaps recreate with FFastrans some atom files for avid change things and Avid isn't able to find again his files. To be honest, I never try that you did with AVid

-->>>Anyway, in this case you have another better method to relink your files :
- Select your old files offline
- Choose batch re import
- select your files no atom (ex op1a)
- Select import and your files are online again

[Ghtais]

Hi

@momocampo : thanks for your reply, I see exactly what you mean but this is not my issue. After the scan I don't need to drag and drop the mdb file again cause I have already clips in my bin. Clips are off line so I need to relink to new media coming from FFAStrans. Theoretically, I don't need to relink but we know that very often avid needs some help to see new clip

@Encodem : I have already read and read again this thread talking about unique ID to be sure that I understand clearly the issue with BMX but I don't understand how it could be involved to my problem. If the 2 files (the first imported and the seconde re-imported) have the same unique ID it should work better than if they have different one.

Anyway I have downloaded your BMX version and I will try it very soon to see if it solves the problem
If you have other ideas don't hesitate.

thank you

[momocampo]

Hey Ghtais,

I'm so sorry I believed your message was from Encodem...Really not awake today...
Sorry again.

Cheers.

[emcodem]

Well using this new BMX build, the UMID's are genereated uniquely and before the UMIDS consisted of something like %TAPENAME%REELNAME%%CURRENTTIME%. So currently you are working with UMIDs that are nearly the same except the last few letters/numbers. Using the new version the UMIDS change completely.
I have a feeling that the behaviour changes with the new UMID type. Maybe not change to be good but i need to know if it changes.

2 files (the first imported and the seconde re-imported) have the same unique ID it should work better than if they have different one.

You are talking about 2 files where each of them has been transcoded with the same package name, reel etc... but they were transcoded at a different time of day? - in that case they should not have the same MP UMID. What makes you think they have the same UMID and are you talking about the Material package UMID?

[Ghtais]

Hi Emcodem

Very interesting !
Because my workflow in FFAStrans never change, I always transcode with the same "Project name" and "Tape name" as I describe upper.
You say UMIDS consisted of something like %TAPENAME%REELNAME%%CURRENTTIME%. Is "current time" only hours and minutes ? or is it a full date with day / month / year and seconds ? If it is simple date like hh:mm, it is possible that 2 different files can be transcoded at the same hour but in a different day ... if they have the same UMIDS Avid can confuse 2 files even if they have different name, no ?

We have a huge amount of files to be ingested and sometimes FFAStrans transcode each night for an entire week. knowing that, one file can be transcode Monday at 3 o'clock in the morning and an other one Tuesday at 3 o'clock in the morning too

I'am not an expert in digital file identification and I don't know exactly how AVID deals with files, clip and link between them. My feeling is that AVID confuse some clip with other even if they have different name so perhaps it should be a trick with the internal identification of the file.

[emcodem]

You say UMIDS consisted of something like %TAPENAME%REELNAME%%CURRENTTIME%. Is "current time" only hours and minutes ? or is it a full date with day / month / year and seconds ?

The date was including everything starting from year to second. The problem in the other thread was about transcoding 2 clips with same tapename in the same second, that is why getting unique umid's solved it for Vinny.
At this moment i don't want to speculate too much but a valid solution for your problem might be to force the umid to match the umid from input mxf (or a checksum of the file in case you don't have an mxf on input).
As far as i understand by now, avid links not only to filenames but the umid plays a big role. We will probably never get out how exactly avid works with the umid stuff, that is one reason why i liked the idea of the umids being totally unique and random.

Could you please test the behaviour with the new BMX tool from the linked thread?

Additional information for further action on this, we need to check for --psp-uid (described as tape/import UID), as here someone hints on avid using this for re-linking.
https://sourceforge.net/p/bmxlib/discus ... #ae56/fb22

[Ghtais]

ok so in my case it's impossible to have the same UMID because I use only one computer with FFAStrans and his clock is synchronized with an universal time service on the internet.
I have installed your BMX patched version. I have ingested some footage this afternoon with no issue. I dare not to try to re import some clip for testing as i have a lots of important works in progress and I don't want to live again an other disaster

As soon as possible I will isolate a part of my network and test ingest / re import but I think it will be very difficult to have good conditions to recreate the issue. I mean, one editing stations with 10 or 20 clips there is no problems, 15 editing stations with 3000 or 4000 clips, it is an other story.

I will update the post
thx all

[emcodem]

Ok, but it sounds important to me to get to the bottom of this. First step in any comlex software issuebis always simolify and reproduce. Maybe there is a chance that you can get out some simple steps how to reproduce the issue in isolated envoronment. I hoped we can reproduce with mcp only, without interplay...