Sampling and Compression

A GPU-Based Real-time Video Compression Method for Video Conferencing

S. Katsigiannis, G. Papaioannou, D. Maroulis, In Proc. 18th IEEE/EURASIP Int. Conf. on Digital Signal Processing (DSP2013), 2013.

Recent years have seen a great increase in the everyday use of real-time video communication over the internet through video conferencing applications. Limitations on computational resources and network bandwidth require video encoding algorithms that provide acceptable quality on low bitrates and can support various resolutions inside the same stream. In this work, the authors present a scalable video coding algorithm based on the contourlet transform that incorporates both lossy and lossless methods, as well as variable bitrate encoding schemes in order to achieve compression. Furthermore, due to the transform utilized, it does not suffer from blocking artifacts that occur with many widely adopted compression algorithms. The proposed algorithm is designed to achieve realtime performance by utilizing the vast computational capabilities of modern GPUs, providing satisfactory encoding and decoding times at relatively low cost. These characteristics make this method suitable for applications like video conferencing that demand real-time performance. The performance and quality evaluation of the algorithm shows that the proposed algorithm achieves satisfactory quality and compression ratio.

Downloads: author-prepared paper version

Practical Frame Buffer Compression

P. Mavridis, G. Papaioannou, In GPU Pro 4 (Ed.: W. Engel), CRC Press, 2013.

In this article we present a method to directly rasterize a full color image using two color channels, instead of three, thus reducing both the consumed storage space and bandwidth during the rendering process. Exploiting the fact that the human visual system is more sensitive to variations of luminance than chrominance, the rasterizer generates fragments in the YCoCg color space and directly stores the chrominance channels at a lower resolution using a mosaic pattern. When reading from the buffer, a simple and efficient edge-directed reconstruction filter provides a very precise estimation of the original uncompressed values. We demonstrate that the quality loss from our method is negligible, while the memory and bandwidth consumption are greatly reduced. Furthermore, the reduction of the output channels results in a sizable increase in the fill-rate of the GPU rasterizer. Our method is trivial to implement, it is compatible with hardware multi-sample anti-aliasing and alpha blending and can be used with both forward and deferred rendering pipelines. Forward pipelines can benefit from the increased fill-rate, while deferred pipelines can use our method to pack more data on a limited number of render targets, something very important on existing consoles. Bandwidth savings are also extremely important on mobile platforms, where memory accesses will drain the battery.

Progressive Screen-space Multi-channel Surface Voxelization

A. Gaitatzes, G. Papaioannou, In GPU Pro 4 (Ed.: W. Engel), CRC Press, 2013.

To alleviate the problems of screen-space voxelization techniques, but maintain their benefit of predictable, controllable and bound execution time relative to full-scene volume generation methods, we introduce the concept of Progressive Voxelization. The volume representation is incrementally updated to include the newly discovered voxels and discard the set of invalid voxels, which are not present in any of the current image buffers. Using the already available camera and light source buffers, a combination of volume injection and voxel-to-depth-buffer re-projection scheme continuously updates the volume buffer and discards invalid voxels, progressively constructing the final voxelization. The algorithm is lightweight and operates on complex dynamic environments where geometry, materials and lighting can change arbitrarily. Compared to single-frame screen-space voxelization, the method provides improved volume coverage (completeness) over non-progressive methods, while maintaining its high performance merits.

A Real-Time Video Encoding Scheme Based on the Contourlet Transform

S. Katsigiannis, G. Papaioannou, D. Maroulis, Design and Architectures for Digital Signal Processing, Dr. Gustavo Ruiz (Ed.), ISBN: 978-953-51-0874-0, InTech, 2013.

In this chapter, a more elaborate analysis of a novel algorithm for high-quality real-time video encoding is presented, based on the original idea in our paper "A Contourlet Transform based algorithm for real-time video encoding". The algorithm is designed for content obtained from low resolution sources like web cameras, surveillance cameras, etc. Critical to the efficiency of video encoding algorithm design is the selection of a suitable image representation method. Texture representation methods proposed in the literature that utilize the Fourier transform, the Discrete Cosine transform, the Wavelet transform as well as other frequency domain methods have been extensively used for image and video encoding. Nevertheless, these methods have some limitations that have been partially addressed by the Contourlet Transform (CT), which our video encoding algorithm is based on. The Contourlet Transform offers multiscale and directional decomposition, providing anisotropy and directionality, features missing from traditional transforms like the Discrete Wavelet Transform. In recent years, the Contourlet Transform has been successfully utilised in a variety of texture analysis applications, including synthetic aperture radar (SAR), medical and natural image classification, image denoising, despeckling of images, image compression, etc. By harnessing the computational power offered by modern graphics processing units (GPUs), a gpu-based contourlet transform is able to provide an image representation method with advantageous characteristics, while maintaining a fast performance.

Downloads: On line chapter   

Reference: BibTex  

The Compact YCoCg Frame Buffer

P. Mavridis, G. Papaioannou, Journal of Computer Graphics Techniques, 1(1), 19-35, 2012.

In this article we present a lossy frame-buffer compression format, suitable for existing commodity GPUs and APIs. Our compression scheme allows a full-color image to be directly rasterized using only two color channels at each pixel, instead of three, thus reducing both the consumed storage space and bandwidth during the rendering process. Exploiting the fact that the human visual system is more sensitive to fine spatial variations of luminance than of chrominance, the rasterizer generates fragments in the YCoCg color space and directly stores the chrominance channels at a lower resolution using a mosaic pattern. When reading from the buffer, a simple and efficient edge-directed reconstruction filter provides a very precise estimation of the original uncompressed values. We demonstrate that the quality loss from our method is negligible, while the bandwidth reduction results in a sizable increase in the fill rate of the GPU rasterizer.

Downloads: paper   video   WebGL demo

Reference: BibTex  

Texture Compression using Wavelet Decomposition

P. Mavridis, G. Papaioannou, Computer Graphics Forum (Proc. Pacific Graphics 2012), 31(7), 2012.

In this paper we introduce a new fixed-rate texture compression scheme based on the energy compaction properties of a modified Haar transform. The coefficients of this transform are quantized and stored using standard block compression methods, such as DXTC and BC7, ensuring simple implementation and very fast decoding speeds. Furthermore, coefficients with the highest contribution to the final image are quantized with higher accuracy, improving the overall compression quality. The proposed modifications to the standard Haar transform, along with a number of additional optimizations, improve the coefficient quantization and reduce the compression error. The resulting method offers more flexibility than the currently available texture compression formats, providing a variety of additional low bitrate encoding modes for the compression of grayscale and color textures.

Downloads: paper (author-prepared pre-print version)   supplemental material   presentation slides

Reference: BibTex  

A Contourlet Transform based algorithm for real-time video encoding

S. Katsigiannis, G. Papaioannou, D. Maroulis, SPIE Photonics Europe, Real-Time Image and Video processing Conference, 16-19 April, Brussels, Belgium, 2012.

(best student paper award)

In recent years, real-time video communication over the internet has been widely utilized for applications like video conferencing. Streaming live video over heterogeneous IP networks, including wireless networks, requires video coding algorithms that can support various levels of quality in order to adapt to the network end-to-end bandwidth and transmitter/receiver resources. In this work, a scalable video coding and compression algorithm based on the Contourlet Transform is proposed. The algorithm allows for multiple levels of detail, without re-encoding the video frames, by just dropping the encoded information referring to higher resolution than needed. Compression is achieved by means of lossy and lossless methods, as well as variable bit rate encoding schemes. Furthermore, due to the transformation utilized, it does not suffer from blocking artifacts that occur with many widely adopted compression algorithms. Another highly advantageous characteristic of the algorithm is the suppression of noise induced by low-quality sensors usually encountered in web-cameras, due to the manipulation of the transform coefficients at the compression stage. The proposed algorithm is designed to introduce minimal coding delay, thus achieving real-time performance. Performance is enhanced by utilizing the vast computational capabilities of modern GPUs, providing satisfactory encoding and decoding times at relatively low cost. These characteristics make this method suitable for applications like video-conferencing that demand real-time performance, along with the highest visual quality possible for each user. Through the presented performance and quality evaluation of the algorithm, experimental results show that the proposed algorithm achieves better or comparable visual quality relative to other compression and encoding methods tested, while maintaining a satisfactory compression ratio. Especially at low bitrates, it provides more human-eye friendly images compared to algorithms utilizing block-based coding, like the MPEG family, as it introduces fuzziness and blurring instead of artificial block artifacts.

Downloads: author-prepared version of the paper   

Reference: BibTex  

High Quality Elliptical Texture Filtering

P. Mavridis, G. Papaioannou, In GPU Pro 3 (Ed.: W. Engel), AK Peters/CRC Press, 2012.

In this chapter, we present a series of simple and effective methods to perform high quality texture filtering on modern GPUs. We base our methods on the theory behind the elliptical weighted average (EWA) filter. We first present an exact implementation of the EWA filter that smartly uses the underlying bilinear filtering hardware to gain a significant speedup. We then proceed with an approximation of the EWA filter that uses the underlying anisotropic filtering hardware of the GPU to construct a filter that closely matches the shape and the properties of the EWA filter, offering vast improvements in the quality of the texture mapping. To further accelerate the method, we also introduce a spatial and temporal sample distribution scheme that distributes samples in space and time, permitting the human eye to perceive a higher image quality, while using fewer samples on each frame.

Texture compression using wavelet decomposition: A preview

P. Mavridis, G. Papaioannou, Proc. I3D 2012: ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games, p. 218 (Poster).

We present a new fixed-rate texture compression scheme based on the energy compaction properties of the Discrete Wavelet Transform. Targeting existing commodity graphics hardware and APIs, our method is using the DXT compression formats to perform the quantization and storage of the wavelet transform coefficients, ensuring very fast decoding speeds. An optimization framework minimizes the quantization error of the coefficients and improves the overall compression quality. Our method provides a variety of low bitrate encoding modes for the compression of grayscale and color textures. These encoding modes offer either improved quality or reduced storage over the DXT1 format. Furthermore, anisotropic texture filtering is performed efficiently with the help of the native texture hardware. The decoding speed and the simplicity of the implementation make our approach well suited for use in games and other interactive applications.

Downloads: poster abstract   

Reference: BibTex  

Two Simple Single-pass GPU methods for Multi-channel Surface Voxelization of Dynamic Scenes

A. Gaitatzes, P. Mavridis, G. Papaioannou, to be presented at Pacific Graphics 2011 (short paper).

Abstract. An increasing number of rendering and geometry processing algorithms relies on volume data to calculate anything from effects like smoke/fluid simulations, visibility information or global illumination effects. We present two real-time and simple-to-implement novel surface voxelization algorithms and a volume data caching structure, the Volume Buffer, which encapsulates functionality, storage and access similar to a frame buffer object, but for threedimensional scalar data. The Volume Buffer can rasterize primitives in 3d space and accumulate up to 1024 bits of arbitrary data per voxel, as required by the specific application. The strength of our methods is the simplicity of the implementation resulting in fast computation times and very easy integration with existing frameworks and rendering engines.

Downloads: author-prepared version of the paper   

Reference: BibTex  

High Quality Elliptical Texture Filtering on GPU

P. Mavridis, G. Papaioannou, Proc. ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games 2011 (I3D), San Francisco, CA, USA, pp. 23-30, 2011.

Abstract. The quality of the available hardware texture filtering, even on state of the art graphics hardware, suffers from several aliasing artifacts, in both spatial and temporal domain. Those artifacts are mostly evident in extreme conditions, such as grazing viewing angles, highly warped texture coordinates, or extreme perspective and become especially annoying when animation is involved.

In this paper we introduce a method to perform high quality texture filtering on GPU, based on the theory behind the Elliptical Weighted Average (EWA) filter. Our method uses the underlying anisotropic filtering hardware of the GPU to construct a filter that closely matches the shape and the properties of the EWA filter, offering vast improvements in the quality of texture mapping while maintaining high performance. Targeting real-time applications, we also introduce a novel spatial and temporal sample distribution scheme that distributes samples in space and time, permitting the human eye to perceive a higher image quality, while using less samples on each frame. Those characteristics make our method practical for use in games and other interactive applications. For cases where quality is more important than speed, like GPU renderers and image manipulation programs, we also present an exact implementation of the EWA filter that smartly uses the underlying bilinear filtering hardware to gain a significant speedup.

Downloads: author-prepared version of the paper    demo video   conference presentation

Reference: BibTex