Researchers from Tongji University and Microsoft Unveil STLVQE: A Groundbreaking AI Approach to Online Video Quality Enhancement

With an increase in the use of the internet, the demand for high-quality and real-time video content and seamless experiences in applications like video conferencing, webcasting, and cloud gaming has become more pronounced. However, this surge in demand has led to challenges, especially concerning low-latency requirements that push for higher video compression rates. This can often result in a noticeable decline in video quality and adversely affect the overall Quality of Experience (QoE).

Researchers have conducted thorough research to address the limitations of existing quality enhancement methods. Finally, a group from Microsoft Research Asia and Tongji University have formulated a technique called STLVQE. It is the first to investigate the issue of improving online video quality and offers the first technique for attaining real-time processing speed.

Conventionally, Online Video Quality Enhancement (Online-VQE) is used. This approach aims to elevate real-time streaming video quality while mitigating the defects caused by aggressive compression algorithms. However, online VQE faces two primary challenges compared to traditional offline VQE methods.

Firstly, they need high-resolution videos in real time. This requirement ensures a smooth viewing experience, making the enhancement process more demanding. Secondly, online video processing techniques must contend with uncontrolled latency, preventing the reliance on future frames for inference. Relying only on current and previous structures introduces potential delays in the overall video playback.

STLVQE does not have these limitations and represents a groundbreaking step toward achieving real-time processing speeds. This design cut down on unnecessary steps in calculating features, making the network’s decision-making process much faster. The key elements of the network, including how it spreads information, lines up details and enhances the overall output, are reworked to minimize repetitive tasks in figuring out these important features.

The researchers emphasized that introducing a distinctive ST-LUT structure is a key aspect of the STLVQE method. This structure helps to fully utilize the temporal and spatial information present in videos, offering a novel way to improve video quality instantly. During the inference phase, the propagation module selects the reference frame and accesses relevant information, which is then processed by the alignment module. Finally, the aligned and preliminarily compensated structures are input into the enhancement module to obtain the final results.

Researchers evaluated the performance of this system and found that STLVQE outperformed widely used single-frame and efficient multi-frame methods. The technique showcased its ability to process 720P-resolution videos in real-time. Also, STLVQE performed comparably with methods intended for higher delays—typically unsuitable for tasks requiring online video quality enhancement—and outperformed most methods for low delays in video quality enhancement.

STLVQE method is a pioneering solution to the challenges posed by real-time online video quality enhancement. In the ever-evolving realm of online applications, STLVQE is a prominent guide in pursuing superior video experiences characterized by high quality and minimal delays. It addresses the limitations of current techniques and introduces innovative approaches to extract and utilize features, marking a noteworthy advancement in the field.

Check out the Paper. All credit for this research goes to the researchers of this project. Also, don’t forget to join our 33k+ ML SubReddit, 41k+ Facebook Community, Discord Channel, and Email Newsletter, where we share the latest AI research news, cool AI projects, and more.

If you like our work, you will love our newsletter..