Scalable Multimedia Streaming

From NSL

Jump to: navigation, search

The Internet continues to change many aspects of our everyday life, including the way we communicate, learn, conduct business, and even entertain ourselves. With an increasing fraction of the population having access to high-speed network connections and the abundance of low-cost, powerful, computing devices, more services are continually made possible over the Internet. One such service that has recently been in high demand is multimedia communication in various forms, including on-demand streaming, live broadcasting, video conferencing, and collaborative virtual environments. For example, YouTube was reported to be the fastest growing web site in the Internet history; currently serving 200—300 million video sessions per day.

Participants in multimedia communication systems typically seek the highest possible quality, and the more they get used to multimedia services, the higher the quality they will expect and demand from the system. These participants, however, use quite heterogeneous computing resources. Heterogeneity arises from many sources, including network bandwidth, processing capacity, network delay, display resolution, and energy constraints. Supporting such heterogeneity is a challenging task. Current systems typically use nonscalable coding techniques for video streams. Therefore, they either degrade the quality for all participants to accommodate the least-powerful one, or they offer a very few (2—3) versions of the same stream to provide limited scalability. This is clearly insufficient in increasingly heterogeneous environments. Furthermore, nonscalable coding systems cannot fully utilize client resources, nor can they maximize the quality for all clients.

Unlike the case with nonscalable coders, various representations of the same video stream with different bit rates and qualities can be produced using scalable coders. Scalable coding techniques thus have the potential to support wide range of clients. A number of research challenges, however, need to be addressed to realize this potential. This goal of this project is to analyze and understand scalable coding techniques, and to design several optimization and streaming algorithms to make the best possible use of them in real multimedia systems. This will enable faster and wider adoption of scalable coding techniques in various multimedia communication systems, which will, in turn, yield better quality for users, and more efficient utilization of network and server resources. We consider scalability along several dimensions: quality (PSNR), power consumption, computation complexity, and multi-views (streams). We are also interested in efficient manipulation of scalable streams on special hardware devices (GPUs).



  • Yuanbin Shen (MSc student)
  • Shabnam Mirshokraie (MSc student)

On-going Research Problems


Software and Data

We have augmented the MPEG-4 FGS reference software to collect rate-distortion information at encoding time. We have also developed scripts (in perl and/or Matlab) to analyze this information and to generate rate-distortion meta data. Using the rate-distortion meta data, we design several bit-allocation algorithms for streaming applications (see our paper for details). We provide our programs and data in the following tar files:

  • Programs: contains the augmented reference software and scripts.
  • Data: contains experimental data for three video sequences: Foreman, Mobile, and Bee.
  • Howto.txt: contains the instructions of how to use our software.
  • Plots_Howto.txt: contains a list of Matlab commands that were used to generate figures from experimental data.