GOP header timecode

[Ghtais]

Hi guy !

not really a FFAstrans question but perhaps some of you can help.
My broadcaster ask me to add GOP header timecode to file that I deliver to him.
The file is XDcam 50 Mbits exported from AVID and rewrapped in AS10 using BMXtranswrapp in FFAStrans
The question is : do you know how to change this GOP header timecode in the file without re encoding with FFmpeg ?
I have discovered ReStream, a little freeware which seems to do the job without re encoding so it seems to be possible !

Thank you

[emcodem]

Hey ghtais, good to hear from you!
Before i answer i want to ask you if you are aware that your customer wants something really special from you.
MXF standard SMPTE 377M makes one point very clear: all metadata of the existing essence (e.g. the GOP timecode) MUST be ignored by the decoder in case the MXF stream carries another metadata value. In other words, whatever timecode the essence stream carries in the GOP or any other timecode, MUST be ignored from your customers decoder. This is for example why some Sony cameras write all GOP TC to zero in each and every GOP header.

So my question is: are you aware that your customer wants from you to deliver something that exceeds all known standards?

[Ghtais]

HI Emcodem

I must say that I was hoping hear you on this subject
My brodcaster says : Every timecode must be growing, continous and identical. I deliver MXF files starting at 01:00:00:00. the GOP header is at 00:00:00:00 so it doesn't be compliant with that.
BATON reports : Serious si Mismatch in GOP Header and Source Package Start TimeCodes.



I don't know exactly what norm or standard they are referring.
What do you thing ?

thank you

[emcodem]

Ah, i just see its about AS11 instead of RDD9, i dont know the details of AS11 at all unfortunately.
Well this TC stuff is not 100% standardized in the underlying smpte papers, so after all it's left over for the application to decide what's best.
I just read in ebu-R122 that the TC should either completely be zero or if not, the essence TC should match the other TC'S in the container. OK, this is just a recommendation but we can also learn from it that the base standards do not mention about that.

OK, anyway i don't yet have a nice commandline way to do rewrite the GOP TC without reencoding the whole file. What we can easily do is to rewrite the first GOP TC to the needed value and leave the rest, this might be compliant with your customer's needs but it is not really desireable i guess hehe.
It would maybe also be pretty simple to set all GOP headers to TC zero but again this is not what we want
Let me investigate what options we have here...

[Ghtais]

I think we can try the "rewrite only the first GOP TC" to 01:00:00:00 and see if it can do the trick
I mean, the goal is to pass BATON test. I think nobody is using this at all.

I would like to avoid reencoding the file to keep the best quality possible. Knowing that it is XDCam and I read before in "XDCAM Encode / Blockiness" tread that It can give some artefacts.

thanks emcodem

[emcodem]

Hehe well, if you want to try
If we just need to replace a single occurence, in mpeg2 this is very easy.

Download gsar:
https://sourceforge.net/projects/gnuwin ... &download=

As you see, i extraced it to c:\temp but of course you want to place it somewhere else for producton.
FIrst try with gsar (@FranceBB no need to warn for slow processing here, this time we dont need cat or type - as we only replace a single thing):

Code: Select all

C:\temp\gsar.exe -s:x00:x00:x01:xB8:x00:x08:x00 C:\temp\1.mxf

this should say "1 occurence found". If it says anything different, no need to go on. (maybe you want to add this with a conditional for checking that all is ok in your workflow later on).

Now the full thing, beware as the original input file will be changed by that:

Code: Select all

C:\temp\gsar.exe -s:x00:x00:x01:xB8:x00:x08:x00 -r:x00:x00:x01:xB8:x04:x08:x00 -o C:\temp\1.mxf

you see? ..we just replcaed one 0x00 by 0x04 and thats the difference
To explain: 000001B8 sequence is the GOP header Start code. The full sequence for a closed GOP, non dropframe, along with a zero start timecode is "000001B8000800" same thing but with start tc to one hour: 0001b8040800

It is not 100% guaranteed that the byte sequence 000001B8000800 only occurs once in your file but 000001B8 is a protected MPEG2 start code, so the chances should be minimal that it occurs accidently somewhere in the mxf metadata. That is why when you work with this, you should definitely implement the first command that only searches (-s) but does not change the file and see if it outputs "1 occurence". Otherwise the mxf strucutre will get damaged.

[Ghtais]

Amazing Emcodem !!

thank you so much to take the time to explain and to share with us your knowledge, it is very appreciated.
So quickly I have tested a few things :
First I have tested a reStream file with Gsar and it gives me one occurrence with x00:x00:x01:xB8:x04:x08:x00. So we can say that it is the same thing.
secondly mediainfo gives me the right information after replacement with Gsar



Next week I will send a test file to my broadcaster for validation. I will post an update as soon as possible

Bye

[Ghtais]

Hi all

I'am coming back for an update.
The broadcaster said : refused

The first GOP is at 01:00:00:00 but for each new GOP, timecode revert to 00:00:00:00 during 1 frame;
he found an other issue that he didn't mention before : GOP is open instead of closed.

humm it sounds the end of FFAstrans for broadcast master.
@emcodem, what do you thing ?

bye

[FranceBB]

GOP open can be easily addressed by switching to Closed and I'll send you the command first thing tomorrow as soon as I get at work (I'm with my mobile now).
About the timecode, I'll leave it to Emcodem.

[emcodem]

Yeah closed GOP xdcamhd stream is something that ffmpeg cannot do or at least we are not able to get it done. The only option to go on with ffastrans would be to introduce another commandline or api encoder, e.g. some mainconcept tool or such. Anyway, i don't know any freely available tool to create closedGOP XDCAMHD compaible mpeg2.

Need to mention that closed GOP is definitely no xdcam commpatibility requirement, as any hardware encoder will usually deliver openGOP, thats Sony cameras, Omneon Servers, K2 Servers and so on...

Would you need to buy something new or have you got some existing solution you can utilize for encoding?

[edit] if they toss the closed gop requirement, i think i could help but it would take some time; it's about a day of work to add the missing function to the ffmpeg mpeg2 bitstream filter.