Versatile Video Coding (VVC) has emerged as the latest and arguably the greatest codec so far, but is being caught up through the same licensing quagmire which has held back deployment of its predecessor, HEVC (High Efficiency Video Coding). It has established a clear lead over rivals in benchmarks for efficiency and bit rate reduction but even its advocates concede it will not gain widespread deployment unless royalty issues are satisfactorily resolved.

There can also be confusion within the name, since VVC was proposed as a successor to HEVC early in 2021 by the Joint Video Exploration Team (JVET), setup for this function by MPEG and the ITU. The work and reference software was called Joint Exploration Model (JEM), but then renamed VVC to mirror improved versatility and the ability to take greater advantage of varying picture complexity for both inter and intra frame compression.

The new codec continues to be at an early stage of development, with completion scheduled for the end of 2021. In setting that timescale, MPEG and ITU are breaking with the tradition of bringing out new codecs about every 10 years, each roughly doubling efficiency or halving bit rate for a given video quality. VVC is scheduled to reach around six years after HEVC was introduced, reflecting accelerating interest in compression efficiency driven by two factors. The first is the increase in streaming of both live and on-demand content over broadband networks with varying bit rate constraints. The second is the growth in Ultra HD content at 4K resolutions, using the prospect better frame rates exerting much more pressure on bandwidth.

MPEG and also the ITU were also motivated through the groundswell behind AV1, the open source and supposedly royalty free codec developed as a successor to Google's VP9 by the Alliance for Open Media (AOM). This powerful consortium's governing members include Amazon, Apple, ARM, Cisco, Facebook, Google, IBM, Intel, Microsoft and Netflix.

AV1 had been edging ahead of HEVC in some performance benchmarks, while gaining traction through its commitment of being royalty free. Furthermore, to counter the potential of some patent infringement cases being brought against AV1, the AOM has set up a legal defence fund to get the tab if any AV1 users are used.

Against this backdrop, JVET felt it had to move fast with VVC and, in the technical perspective, has succeeded in setting up a powerful marker, even while the codec is still under development.

Technically, VVC is a progression from HEVC with some significant enhancements. Her same basic partitioning structure by which input video is divided into blocks called Coding Tree Units, which in HEVC were extended to permit 64×64, 32×32 or 16×16, having a larger pixel block size increasing coding efficiency. These CTUs are in turn cleaved into a number of coding units (CUs). One of the improvements in VVC is larger flexibility in this sub-structure of CUs, that is now arranged in a nested-tree like arrangement, allowing each CU to become rectangular as well as square, for instance.

While such fundamental architectural facets of VVC have been defined, specific information on the codec syntax, covering aspects such as sequence of events during execution, in addition to detailed partitioning of the image into CTUs in the higher level, haven't yet been determined. That being the case, VVC's early benchmark figures are remarkably good, with recent tests conducted by the BBC indicating a 27% decrease in bit rate compared with HEVC for any given video quality and 25% over AV1. This is about in line with several other tests and suggests that the target of 50% reduction over HEVC is well within reach.

Most codec experts agree that technically VVC is on the right track, one being Ken McCann, Founder and Director at UK-based technology consultancy Zetacast and former Chairman from the DVB technical group responsible for defining audio-visual coding standards. McCann, though, argues the behaviour by some licence holders has rendered HEVC ill-suited with a key market segments, for example video streaming applications, and that exactly the same doubts hang over VVC. In contrast to its predecessors – H.264 and before that, MPEG-2 – HEVC is covered by three competing patent pools: from MPEG-LA, HEVC Advance and Velos. This presents a confusing and risky picture for potential users.

Another streaming consultant, Jan Ozer, who specialises within the application of H.264, H.265 and VP9 encoding for live and on-demand production, goes further. He warns of VVC struggling with \”HEVC syndrome\” towards the extent that it makes no sense for MPEG to produce another codec without resolving what he calls the \”royalty land grab\” that is still going on over 5 years following the launch of HEVC. This then, is really a more urgent and difficult challenge for MPEG than the technical certainly one of pushing the boundaries of video compression further.