Скачать или смотреть Zoe Liu - Decoder Complexity Aware AV1 Encoding Optimization

AV1 has included more than 100+ coding tools compared to its predecessor VP9. AV1 hence left an impression to the community that yes, it is more coding efficient, but it is slow, fairly slow. Indeed, great thanks to our open source communities, AV1 encoders have been constantly optimized w.r.t. both its coding efficiency and encoding speed. To date, there are three well known open-source AV1 encoders: SVT-AV1 initiated by the joint effort between Intel and Netflix, and has also been specified by AOM SIWG as the version-0 codebase; rav1e initiated by Mozilla, as well as libaom initiated by AOMedia. For instance, from January 2019 through April 2020, just checking libaom by example, the AV1 encoder has been speedup by more than10x (not 10%), and the coding efficiency, quantified by the well acknowledged BD-rate figures, has also been improved by more than10% concurrently.

Also, sincere thanks to the AOMedia funded dav1d endeavor, an open AV1 software decoder initially jointly developed by VideoLan, Mozilla, and Two Orioles, the launching of AV1 as real products within 2 years of the finalization of the standard is not a far-fetched dream anymore, before the wide availability of hardware decoders in the market. The continuous optimization of dav1d opens the door to AV1, and feasible user scenarios are being identified where software decoding solutions can be deployed and accepted.

In this talk, we will address the AV1 encoding optimization from a different angle, namely decoder complexity aware AV1 encoding optimization. As we hone our strategies and algorithms to optimize AV1 encoders, in general, we are all trying on more superior coding efficiency well outperforming existing solutions, and meanwhile accelerating the encoding speed ready for a real product. Indeed, the awareness of the decoder complexity can in return collect valuable feedbacks to the AV1 encoder design, so that an AV1 encoder not only can achieve a sufficient good tradeoff between coding efficiency and encoding speed, but also help release the burden on the decoder side, to make the deployment of AV1 to an even wider community.

We will first have a brief overview on current performance for dav1d, including stats collected specifically regarding its decoding speed and power consumption. The decoding speed of dav1d has been consistently confirmed to outperform all other existing AV1 software decoders. To check its performance over mobile phones, we have evaluated dav1d against the ffmpeg-h264 software decoder and the openhevc software decoder, in terms of CPU usage [%], memory using [MB], current [mA], consumed power[mW], voltage [mV], and temperature [℃], over several typical mobile devices. Our results demonstrate that power consumption wise, dav1d is ranked between the aforementioned two decoders, worse than ffmpeg-h264 but better than openhevc.

Furthermore, we worked with our collaborators, to collect extensively the decoding speed stats running dav1d particularly over low end mobile devices. For instance, our collaborator selected 18 common test sequences, used SVT-AV1 Preset 6 to generate AV1 bitstream, and decoded them on XiaoMi 4, with Qualcomm Snapdragon 801 Quad-core processor. The test set of videos include resolution of 720x536, 960x536, 960x480, 1024x576, and 2024x536. It can be observed that at higher encoding bitrate, the use of decoder on such types of devices would be very challenging to decode in real time.

Then we will address several coding tools that may directly affect the performance of the decoder complexity. We will then demonstrate the results that for specific content, certain coding tools may incur a large complexity at the decoder side whereas the encoding efficiency may only suffer a minimum degradation if such tools are skipped at the encoder side. For instance, AV1 includes block sizes ranging from 4x4 all the way to 128x128. By turning off extremely large partitions or the smallest partitions, a good trade-off may be achieved for a sufficiently good coding efficiency while incur not too much decoder side computational complexity, thus to have the decoder more capable of decoding in real time even over low end devices. The decoder complexity aware encoding design concept would further help drive the launching of AV1 forward.

Presented at Demuxed 2020.