1. stream structure
2. Sequence headers
3. Sequence Extension
4. Sequence Display Extension
5. Quant Matrix Extension
6. Copyright extension
7. Sequence Scalable Extension
8. Group of Pictures
9. Picture headers
10. Picture Display Extension
11. Picture Coding Extension
12. Slice

The structure differs from that of MPEG-1 elementary video in terms of the possible extensions and, in the case of the TMPG encoder, the number of slices.

 
The video stream consists of several consecutive sequences.

Sequence:

 
The sequence can or should contain further sequence headers. Usually before each Group of Picture (GOP). The sequence headers are needed for creating entry points (chapter markers) and for fast scrolling. At the end there should be a sequence decode. However, this does not seem to be common with MPEG-2.

Group of Picture (GOP):

 
The GOP contains the different picture types. There are I, P, B and D frames.

Intra picture (I-frames) are full images. Predicted picture (P frames) are partial pictures and refer to the previous I/P frames. Bidirectional pictures (B frames) are also partial pictures and refer to the preceding and following I/B frame. Direct Coded Picture (D-Frames) are not used in MPEG-2.

Picture:

 
The order of the images does not correspond to the order in which they are displayed. The images are composed of slices.

Slice:

 
Slices are a vertical and horizontal aggregation of macroblocks.

macro block:

 
A macroblock comprises an image section with a total of 16x16 picture elements (pixels). It stores the YUV 4:2:0 (YCbCr) color information. The Y components of all pixels are contained in the first four blocks b0 to b3. In the fifth block b4, the blue chrominance values ​​(Cb) are stored in one point for each 4 pixels. The corresponding red chrominance values ​​(Cr) are in the sixth block b5.

Each header starts with the PACK_START_CODE_PREFIX, which consists of the three bytes 0, 0 and 1. As a hexadecimal number, it is represented as $000001. This is followed by the ID. For the sequence header, this is the value $B3. The length of the header has different lengths according to the matrices used.

Each sequence header represents a possible entry point. This means that a chapter can only be created where there is a sequence header.

It should also happen that sequence headers are not marked with $000001B3 but with $000000B3.

1. 4 bytes: SEQUENCE_HEADER_CODE = $000001B3
2. 12 Bit: Width - Image width in pixels
3. 12 Bit: Height - image height in pixels
4. 4 bit: Aspect Ratio - aspect ratio
5. 4 bit: frame rate - refresh rate
6. 18 bit: bit rate
7. 1 bit: markers
8. 10 bits: VBV
9. 1 bit: Constrained parameter flag
10. 1 bit: load intra matrix (0) or standard (1)?
11. 64 bytes: Intra matrix if non-standard matrix is ​​used.
12. 1 Bit: Load Non Intra Matrix (0) or Standard (1)?
13. 64 bytes: Non Intra Matrix, unless Standard Matrix is ​​used.

As a scheme:

The bit rate is given in 400 bits per second. The value $3FFFF is supposed to mean a variable bitrate. In the test, TMPGEnc and CCEB each indicated the maximum bit rate there.

The marker bit is intended to detect errors. It must always have the value 1.

The VBV is the memory required to decode the images. It is specified in 16 kB blocks.

The constrained parameter flag is always set to 0 here. The TMPG Encoder 2.5* uses the 10 bits for the VBV and the bit of the constrained parameter flag for the VBV specification, so that only half of the VBV is displayed here.

The intra or non intra matrix load bit indicates whether the standard matrix is ​​used or whether the corresponding matrix is ​​saved at this point and must be loaded. The matrix is ​​stored in Zig Zag Scan.

At least one extension follows the sequence header. An MPEG-2 is identified via this. The ID of the extension is $B5. This is followed by the ID of the extension. These extensions should be considered for the sequence header:

1. Sequence Extension
2. Sequence Display Extension
3. Quant Matrix Extension
4. Copyright Extension
5. Sequence Scalable Extension

Extensions have the ID $B5. This is followed by the ID of the extension, here $1.

1. 4 bytes: EXTENSION_START_CODE = $000001B5
2. 4 bits: Start Code Identifier - Sequence Extension = $1
3. 4 bit: profile
4. 4 bits: levels
5. 1 bit: progressive sequence
6. 2-bit: chroma format
7. 2 bit: wide extension
8. 2 bits: height extension
9. 12-bit: bit rate extension
10. 1 bit: markers
11. 8-bit: VBV Buffer Extension
12. 1 bit: low delay
13. 2 bits: Frame rate extension numerator
14. 5 bits: Frame rate extension denominator

As a scheme:

Extensionen are ID $B5. The ID is $2.

4 Byte: EXTENSION_START_CODE = $000001B5

4 Bit: Start Code Identifier - Sequence Display Extension = $2

3 Bit: Video Format

1 Bit: Color, if set:

1. 1 Byte: Color Primaries
2. 1 Byte: Transfer Characteristics
3. 1 Byte: Matrix Coefficients

14 Bit: Display Width

1 Bit: Marker

14 Bit: Display Height

As a scheme:

Explainations

Video Formats are:

Color Primaries and Transfer Characteristics are:

Matrix Coefficients:

Extensions have the ID $B5. The ID of this extension is $3.

4 bytes: EXTENSION_START_CODE = $000001B5

4 bits: Start Code Identifier - Quant Matrix Extension = $3

1 bit: Load Intra Quantiser Matrix, if 1 then

• 64 bytes: intra quantizer matrix

1 Bit: Load Non Intra Quantiser Matrix, if 1 then

• 64 bytes: non-intra quantiser matrix

1 bit: Load Chroma Intra Quantiser Matrix if 1 then

• 64 bytes: Chroma Intra Quantizer Matrix

1 bit: Chroma Non Intra Quantiser Matrix, if 1 then

• 64 bytes: Chroma Non Intra Quantizer Matrix

As a scheme:

Extensions have the ID $B5. This extension has the ID $4.

• 4 bytes: EXTENSION_START_CODE = $000001B5
• 4 bits: Start Code Identifier - Copyright Extension = $4
• 1 bit: copyright flag
• 8 bits: copyright identifiers
• 1 bit: original or copy
• 7 bits: Reserved
• 1 bit: markers
• 20 bit: copyright number 1
• 1 bit: markers
• 22 bit: copyright number 2
• 1 bit: markers
• 22 bit: copyright number 3

As a scheme:

Extensions generally have ID $B5. This extensions has got ID $5.

• 4 Byte: EXTENSION_START_CODE = $000001B5
• 4 Bit: Start Code Identifier - Sequence Scalable Extension = $5
• 2 Bit: Scalable Mode (Data partioning, Spartial, SNR, Temporal)
• 4 Bit: Layer ID

Ist der Scalable Mode “spatial scalability”

• 14 Bit: Lower Layer Prediction Horizontal Size
• 1 Bit: Marker
• 14 Bit: Lower Layer Prediction Vertical Size
• 5 Bit: Horizontal Subsampling Factor M
• 5 Bit: Horizontal Subsampling Factor N
• 5 Bit: Vertical Subsampling Factor M
• 5 Bit: Vertical Subsampling Factor N

Ist der Scalable Mode “temporal scalability”

1 Bit: Picture Mux Enable, if 1 then

• 1 Bit: Mux To Progressive Sequence

3 Bit: Picture Mux Order

3 Bit: Picture Mux Factor

As a scheme:

Scalable Mode = "spatial scalability":

Scalable Mode = "temporal scalability":

The ID $B8 follows the PACK_START_CODE_PREFIX for the GROUP_START_CODE. The length of the header is 4 bytes.

• 4 bytes: GROUP_START_CODE = $000001B8
• 1 bit: drop frame
• 5 bit: Time code hours from 0 to 23.
• 6 bit: Time code minutes from 0 to 59.
• 1 bit: markers
• 6 bit: Time Code seconds from 0 to 59.
• 6 bit: Time Code Picture from 0 to 59.
• 1 bit: Closed GOP
• 1 bit: broken link

As a scheme:

Explanations

The drop frame is used for NTSC to lower the frame rate from 30 to 29.97 fps.

The marker must again be set.

A GOP is closed if the pictures in a GOP relate only to pictures in its own GOP. Even if the stream is not encdet closed, the first and penultimate (CCE Basic) or last (TMPG Encoder) GOP are closed.

The broken link flag is set when frames refer to frames that no longer exist after a cut.

Picture Header

PACK_START_CODE_PREFIX follows PICTURE_START_CODE with ID $00.

Construction

• 4 Byte: PICTURE_START_CODE = $00000100
• 10 Bit: Temporal Reference
• 3 Bit: Coding Type
• 16 Bit: VBV Delay.

P- and B-Frames:

• 1 Bit: full fel forward vector
• 3 Bit: forward f code

B-Frames only:

• 1 Bit: full backward vector
• 3 Bit: backward f code

Extra Information:

• 1 Bit: Extra Info Bit Set
• 1 Byte: Extra Information, if Extra Bit is set.
• 1 Bit: Extra Info Bit cleared

Als Schema

Erläuterungen

The temporal reference is the order in which the images should be displayed. The first picture of the group has the value 0.

Coding types:

• Resticted
• I - Intra Coded
• P - Predictive Coded
• B - Bidirectionally Predictive Coded
• D - DC Intra Coded
• Reserved
• Reserved
• Reserved

The VBV delay is specified in 90 kHz cycles at constant bit rates. With variable bit rate, the delay is set to FFFF. I am missing further information about the forward and backward vectors and the extra information. The extra information consists of nine bits. The first is the indicator for extra information to follow. The following 8 bits represent the extra information.

Extensions

With MPEG-2, the picture header is followed by various extensions. The ID of the extension is $B5. That concludes the ID of the extension. These extensions should be considered for the picture header:

• 7 - Picture Display Extension
• 8 - Picture Coding Extension
• 9 - Picture Spatial Scalable Extension
• 10 - Picture Temporal Scalable Extension

The extension start code with the ID $B5 is followed by four bits with the extension ID $B7.

• 4 bytes: EXTENSION_START_CODE = $000001B5
• 4 bits: Start Code Identifier - Picture Display Extension = $7
• For 0 = i < number_of_frame_centre_offsets (maximum 3 offsets)
  • 2 bytes (Int16): frame_centre_horizontal_offset
  • 1 bit: markers
  • 2 bytes (Int16): frame_centre_vertical_offset
  • 1 bit: markers

As a scheme:

Explanations

This extension is not necessary for the normal decoding process. However, it allows the picture to be positioned on the display. An application for this is Pan & Scan . This extension must be preceded by a sequence display extension.

frame_centre_horizontal_offset
Is the horizontal offset in units of 1/16 sample. A positive value means the center of the picture is to the right of the center of the display.

frame_centre_vertical_offset
Is the vertical offset in units of 1/16 sample. A positive value means the center of the picture is below the center of the display.

The size of the display is defined in the Sequence Display Extension. The coordinates for the decoded picture in the picture display extension. The center of the decoded picture is the center of the display. A picture can refer to one, two or three decoded fields, ie it can contain up to three offsets.

The number of offsets results from the flags progressive_sequence, repeat_first_field and top_field_first.

if ( progressive_sequence == 1)
   { if ( repeat_first_field == 1 )
        { if ( top_field_first == 1 )
             number_of_frame_centre_offsets = 3
          else
             number_of_frame_centre_offsets = 2
        }
        else
        {
        number_of_frame_centre_offsets = 1
        }
   }
   else
   { if (picture_structure == "field")
        { number_of_frame_centre_offsets = 1
        }
        else
        { if (repeat_first_field == 1 )
             number_of_frame_centre_offsets = 3
          else
             number_of_frame_centre_offsets = 2
        }
   }

The progressive_sequence flag is included in the sequence extension. repeat_first_field, top_field_first, and picture_structure in the Picture Coding Extension.

The absence of the offsets means that the previously used values ​​should be used. That also applies, if not all offsets are given. After a sequence header, null values ​​are used again until values ​​are specified again.

The extension start code with the ID $B5 is followed by four bits with the extension ID $B8.

4 bytes: EXTENSION_START_CODE = $000001B5

4 bits: Start Code Identifier - Picture Coding Extension = $8

4 bits: f_code[0][0] - forward horizontal

4 bits: f_code[0][1] - forward vertical

4 bits: f_code[1][0] - backward horizontal

4 bits: f_code[1][1] - backward vertical

2-bit: Intra DC precision

2-bit: Picture Structure

1 bit: top field first

1 bit: Frame Pred Frame DCT

1 bit: Concealment Motion Vectors

1 bit: Q scale type

1 bit: Intra VLC format

1 bit: alternate scan

1 bit: Repeat First Field

1 bit: Chroma 420 type

1 bit: progressive frame

1 bit: Composite Display Flag, if set:

• 1 bit: V axis
• 3 bits: field sequence
• 1 bit: Subcarrier
• 7 bits: burst amplitude
• 8 bits: sub-carrier phase

As a scheme:

Explanations

f_code[s][t]
The values ​​are used to decode the motion vectors. The value 0 is forbidden, 1 to 9 and 15 are allowed.

Intra DC Precision
Accuracy with which the discrete cosine transformation is used. The Intra DC Mult(iplikator) is derived from this.

Picture Structure

If a frame is encoded in fields, there must always be a pair with the same picture coding type.

Top Field First
This flag depends on Picture Structure, Progressive Sequence and Repeat First Field.
At If the Progressive Sequence flag is not set, it is used during decoding to reconstruct the frame and indicates whether the First Field (upper field) is output first.
If the Progressive Sequence flag is set, there is in connection with the Repeat flag First Field indicates how often a frame is output during decoding. This means at Repeat First Field = 0 and Top Field First = 0 becomes a progressive frame, with Repeat First Field = 1 and Top Field First = 0 two identical progressive frames become and with Repeat First Field = 1 and Top Field First = 1, three identical progressive frames are output.

Frame Pred Frame DCT
If this flag is set, only Frame DCT and Frame Prediction will be used. At Field Picture (fields) it is 0, with progressive frames it is 1.

Concealment Motion Vectors
This flag indicates whether the intra macroblocks were encoded with motion vectors.

Q Scale Type
This flag is used for the Quantiser Scale Factor.

Intra VLC format
This flag is used to determine the DCT coefficients.

Alternate Scan
This flag is used to determine the DCT coefficients.

Repeat First Field
If the flags progressisve_sequence (see Sequence Extension Header) and progressive_frame are not set, the repeat_first_field flag is also not set. When decoding then the frame composed of two fields. If the progressive_sequence flag is not set but the progressive_frame flag is set, the frame is composed of two fields. The first field (top field or bottom field) is specified by the flag top_field_first identified and followed by the other. If the repeat_first_field flag is then set, the frame consists of three fields composed. The first field is determined by the top_field_first flag and is followed by the other. Then will repeats the first field. If the progressive_sequence flag is set and the repeat_first_field flag is not, the frame decoded from a frame.

Chroma 420 Type
Set the same as Progressive Frame. Exists for historical reasons.

Progressive Frame
If the flag is not set, it means that the two fields of a frame are two interlaced fields are. Repeat First Field must be 0 (two field duration). If the flag is set, this means that the two fields are combined into one are merged. Picture Structure must be set to "Frame" and Frame Pred Frame DCT to 1.

Composite Display Flag
This flag is set if the picture from which the MPEG was encoded is (analogue) Composite Video were encoded. The information refers to the extension following picture. Is it a Frame Picture, the information relates to the First Field. The information is adjusted for the second field, as these cannot be saved. The following elements are not used for decoding. The Repeat First Field Flag and the Composite Display Flag must not be set at the same time.

V Axis
1-bit integer used only when the bitstream represents a signal that had previously been encoded according to PAL systems. v_axis is set to 1 on a positive sign, v_axis is set to 0 otherwise.

Field Sequence
Returns the field number of an eight field sequence on a PAL system or a five field sequence for an NTSC system according to the following table:

Sub Carrier
If the flag is not set, the sub-carrier/line frequency relationship is correct.

Burst Amplitude
specifies the burst amplitude for PAL and NTSC.

Sub Carrier Phase
Indicates the phase of the reference sub carrier of the field synchronization.

The extension start code with the ID $B5 is followed by four bits with the extension ID $B8.

4 bytes: EXTENSION_START_CODE = $000001B5

4 bits: Start Code Identifier - Picture Coding Extension = $8

4 bits: f_code[0][0] - forward horizontal

4 bits: f_code[0][1] - forward vertical

4 bits: f_code[1][0] - backward horizontal

4 bits: f_code[1][1] - backward vertical

2-bit: Intra DC precision

2-bit: Picture Structure

1 bit: top field first

1 bit: Frame Pred Frame DCT

1 bit: Concealment Motion Vectors

1 bit: Q scale type

1 bit: Intra VLC format

1 bit: alternate scan

1 bit: Repeat First Field

1 bit: Chroma 420 type

1 bit: progressive frame

1 bit: Composite Display Flag, if set:

• 1 bit: V axis
• 3 bits: field sequence
• 1 bit: Subcarrier
• 7 bits: burst amplitude
• 8 bits: sub-carrier phase

Explanations

f_code[s][t]
The values ​​are used to decode the motion vectors. The value 0 is forbidden, 1 to 9 and 15 are allowed.

Intra DC Precision
Accuracy with which the discrete cosine transformation is used. The Intra DC Mult(iplikator) is derived from this.

Picture Structure

If a frame is encoded in fields, there must always be a pair with the same picture coding type.

Top Field First
This flag depends on Picture Structure, Progressive Sequence and Repeat First Field.
At If the Progressive Sequence flag is not set, it is used during decoding to reconstruct the frame and indicates whether the First Field (upper field) is output first.
If the Progressive Sequence flag is set, there is a repeat in connection with the flag First Field indicates how often a frame is output during decoding. This means at Repeat First Field = 0 and Top Field First = 0 becomes a progressive frame, with Repeat First Field = 1 and Top Field First = 0 two identical progressive frames become and with Repeat First Field = 1 and Top Field First = 1, three identical progressive frames are output.

Frame Pred Frame DCT
If this flag is set, only Frame DCT and Frame Prediction will be used. At Field Picture (fields) it is 0, with progressive frames it is 1.

Concealment Motion Vectors
This flag indicates whether the intra macroblocks were encoded with motion vectors.

Q Scale Type
This flag is used for the Quantiser Scale Factor.

Intra VLC format
This flag is used to determine the DCT coefficients.

Alternate Scan
This flag is used to determine the DCT coefficients.

Repeat First Field
If the flags progressisve_sequence (see Sequence Extension Header) and progressive_frame are not set, the repeat_first_field flag is also not set. When decoding then the frame composed of two fields. If the progressive_sequence flag is not set but the progressive_frame flag is set, the frame is composed of two fields. The first field (top field or bottom field) is specified by the flag top_field_first identified and followed by the other. If the repeat_first_field flag is then set, the frame consists of three fields composed. The first field is determined by the top_field_first flag and is followed by the other. Then will repeats the first field. If the progressive_sequence flag is set and the repeat_first_field flag is not, the frame decoded from a frame.

Chroma 420 Type
Set the same as Progressive Frame. Exists for historical reasons.

Progressive Frame
If the flag is not set, it means that the two fields of a frame are two interlaced fields are. Repeat First Field must be 0 (two field duration). If the flag is set, this means that the two fields are combined into one are merged. Picture Structure must be set to "Frame" and Frame Pred Frame DCT to 1.

Composite Display Flag
This flag is set if the picture from which the MPEG was encoded is (analogue) Composite Video were encoded. The information refers to the extension following picture. Is it a Frame Picture, the information relates to the First Field. The information is adjusted for the second field, as these cannot be saved. The following elements are not used for decoding. The Repeat First Field Flag and the Composite Display Flag must not be set at the same time.

V Axis
1-bit integer used only when the bitstream represents a signal that had previously been encoded according to PAL systems. v_axis is set to 1 on a positive sign, v_axis is set to 0 otherwise.

Field Sequence
Returns the field number of an eight field sequence on a PAL system or a five field sequence for an NTSC system according to the following table:

Sub Carrier
If the flag is not set, the sub-carrier/line frequency relationship is correct.

Burst Amplitude
specifies the burst amplitude for PAL and NTSC.

Sub Carrier Phase
Indicates the phase of the reference sub carrier of the field synchronization.

A macroblock has a size of 16 * 16 pixels. It consists of 4 blocks of 8 * 8 pixels with gray values ​​(Y). These are supplemented by two blocks of 8 * 8 pixels with chrominance values. Chrominance values ​​mean color values. One block for Red (Cr) and one for Blue (Cb).

• Addr Incr: number of MBs to skip.
• Type: Does this MB use a motion vector? What type?
• QScale: How is the quantization table scaled in this MB?
• Coded Block Pattern (CBP): bitmap indicating which blocks are coded.