PERFORMANCE

Updated 2018-03-07

The single most important aspect of a video codec is its ability to compress video sequences to as small size as possible with as high visual quality as possible. It is common to compare new codecs to old codecs by reporting how much the bit rate can be reduced while maintaining a certain quality level using a certain quality metric. The reduction can be very different for different sequences and for different bit rates, but one well accepted approach is the Bjontegaard-Delta method which is able to provide a single number representing bit rate reduction for a sequence over a range of quality levels.

The first version of xvc (1.0) that was released in September 2017 was already able to outperform HEVC, and all other video codecs, in terms of compression performance. The next version of xvc which is planned to be released during 2018 is providing even better compression. In preparation for the IETF meeting in March 2018, Divideon generated test results for test sequences and test conditions as defined be the NETVC Working Group of IETF. The description of those test conditions is available here and the Internet Draft about xvc is available here. The full set of results can be found at awcy.divideon.com.

The xvc codec works extremely well under the most challenging conditions; when the bitrate is pushed towards rate points where other codecs break down into a chaos of blockiness and bluriness (which is demonstrated in Divideons online demo). The table below shows the bitrate savings of xvc relative to AV1 for single pass coding of the 360p sequences in NETVC’s objective-1-fast test set.

 PSNR‑YPSNR‑CbPSNR‑CrPSNR‑HVSSSIMMS‑SSIMVMAF
Average-19.7-9.6-3.4-22.5-23.0-26.1-19.4
blue_sky_360p_60f-12.4-3.4-5.0-14.4-15.7-19.7-8.4
kirland360p_60f-33.5-40.4-19.0-42.7-38.9-48.9-44.6
niklas360p_60f-21.1-20.1-23.7-24.9-29.1-29.3-15.6
red_kayak_360p_60f-3.727.469.9-5.4-3.3-5.4-8.4
shields_640x360_60f-25.6-3.4-22.0-23.1-28.2-26.3-17.9
speed_bag_640x360_60f-30.1-20.0-17.5-32.8-32.9-36.9-27.8
thaloundeskmtg360p_60f-11.7-6.7-6.3-13.8-13.1-16.5-13.5

Bitrate savings (%) for xvc relative to AV1 (single pass)

The following graph shows the PSNR and bitrate for the Speedbag sequence for xvc, AV1, HEVC (HM), HEVC (x265) and VP9. It should be noted that the VP9 results are with multi-pass coding while the others are with single pass coding.

The numbers for xvc’s bitrate savings relative to AV1 are quite similar when performing multi-pass encoding:

 PSNR‑YPSNR‑CbPSNR‑CrPSNR‑HVSSSIMMS‑SSIMVMAF
Average-15.9-6.2-11.2-19.9-19.0-21.2-14.4
blue_sky_360p_60f0.1-1.9-9.3-5.3-6.9-10.47.1
kirland360p_60f-25.6-32.2-18.1-32.7-31.3-35.4-31.7
niklas360p_60f-6.79.73.4-10.9-8.4-9.4-4.6
red_kayak_360p_60f-13.82.7138.5-19.3-13.2-17.1-19.2
shields_640x360_60f-6.521.61.4-8.2-10.9-10.11.3
speed_bag_640x360_60f-31.7-22.9-19.8-34.6-35.8-38.0-25.5
thaloundeskmtg360p_60f-26.9-20.2-17.9-28.5-26.6-28.1-28.4

Bitrate savings (%) for xvc relative to AV1 (multi-pass)

Does xvc only provide gains for low resolution videos and/or low bitrate videos?

No, as can be seen in the full set of results, xvc actually outperforms HEVC at all resolutions and in all test categories. Compared to AV1, xvc delivers superior compression across all resolutions and all test categories except for the “Screen Content” category.

The following graph shows the PSNR and bitrate for the 720p sequence RollerCoaster. Results are reported for xvc, AV1, HEVC (HM), HEVC (x265) and VP9. Again, it should be noted that the VP9 results are with multi-pass coding while the others are with single pass coding.

The tables and figures reported on this pages are based on the results reported at awcy.divideon.com. Information about the revision and configuration parameters that have been used are available in the xvc Internet Draft. It should be noted that both AV1 and the second version of xvc are still under development and therefore the reported numbers might not represent the performance difference once these versions have been released. The numbers for x265 and VP9 that are included in the graphs are imported from arewecompressedyet.com.

Mobile device performance

The xvc decoder has been tested on different mobile devices to determine decoding capabilities and impact on battery life.

Most smartphones with at least four cores can run xvc software decoding of HD video in real-time.

In one test, a Samsung Galaxy 8+ smartphone was able to playback SD video with less than 500 kbit/s video for 9 hours and 3 minutes using the xvc reference software decoder. For the same video sequence coded at similar quality with the H.264 codec, the playback could last for 9 hours and 20 minutes, using the H.264 hardware decoder.

In another test the Samsung Galaxy 8+ was able to decode 720p video with H.264 at 800 kbit/s for 8 hours 45 minutes and a xvc codec video at similar quality, at around 400 kbit/s for 7 hours.

In general, our testing indicates that at video bitrates lower than 1 mbps the battery consumption used by the processor becomes almost negligible compared to what is used by the display and the data connection. Further optimizations of the xvc decoder software is expected to lower the battery consumption even more.