Overview of fine granularity scalability in MPEG-4 video standard

Presentation: PowerPoint, PDF

Summary:

Fine granularity scalability (FGS) is an amendment on the MPEG-4 standard. This work aims to optimize the video quality over a given range of bit rate. This was motivated by streaming video over the internet, where two critical assumptions come into play:

• the encoder does not know the channel capacity
• the decoder may not be able to decode all bits received over from the channel (or the bitstream may be truncated)

This requires the bitstream should be partially decodable at any bit rate within the supported bit rate range, which is not supported by some other layered scalable techniques such as signal-to-noise ratio coding, temporal scaling or spatial scaling. The goal was to have the quality at a given bit rate as close as possible to what non-scalable technique would give, which represents the optimum.

The technique uses two layers, a base layer and an enhancement layer. The full base layer is required for the decoding of the video stream, and the enhancement layer is structured such that even a partial bitstream can be used to improve the video quality. This is done using bit-plane coding. A bit-plane is defined by taking one bit at the same significant position from each quantized DCT coefficient (absolute, in binary form). The so-formed bit-planes are encoded using variable length encoding, starting form the most significant bit-plane. This also gives a 20% bit saving over run-length coding.

Some advanced features have also been added to FGS, these are:

• Frequency weighting: bits of visually more important frequency components are included earlier in the stream by shifting those bits into higher bit-planes
• Selective enhancement: bits of selected spatial regions of a frame are placed ahead in the bitstream (similar as in frequency weighting)
• Error resilience: extra resynchronization markers are added for each bit-plane
• FGS temporal scalability: combines FGS with temporal scalability (as separate layer or included in FGS layer)

Two profiles were defined to facilitate FGS in MPEG, one for each layer:

• Advanced Simple Profile (base layer)
• FGS Profile (enhancement layer)

The coding efficiency of FGS was compared to simulcast, SNR, and non-scalable coding. FGS was found to outperform multicast at the low and high end of the bit-rate range, and much more efficient than SNR, particularly at the upper end of the bit-range. Compared to non-scalable, SNR was about 2-dB worse at the high end.

Class discussion:

• Bit-plane coding has been used prior to this work and is not a new innovation
• How to decide which bits are more important than others (frequency weighting and spatial enhancement)?
  • difficult, left to individual implementations of to figure out (encourage competition and innovation)
• another encoding technique is being developed that outperforms FGS, achieving quality very close to non-scalable coding, which would make FGS obsolete
• despite having been added to the MPEG-4 standard several years ago (2001), FGS has not yet been widely used, and given the previous point, it may never be.