Light field is one of the most representative image-based rendering techniques that generate novel virtual views from images instead of 3D models. The light field capture and rendering process can be considered as a procedure of sampling the light rays in the space and interpolating those in novel views. As a result, light field can be studied as a high-dimensional signal sampling problem, which has attracted a lot of research interest and become a convergence point between computer graphics and signal processing, and even computer vision. This lecture focuses on answering two questions regarding light field sampling, namely how many images are needed for a light field, and if such number is limited, where we should capture them. The book can be divided into three parts. First, we give a complete analysis on uniform sampling of IBR data. By introducing the surface plenoptic function, we are able to analyze the Fourier spectrum of non-Lambertian and occluded scenes. Given the spectrum,we also apply the generalized sampling theorem on the IBR data, which results in better rendering quality than rectangular sampling for complex scenes. Such uniform sampling analysis provides general guidelines on how the images in IBR should be taken. For instance, it shows that non-Lambertian and occluded scenes often require a higher sampling rate. Next, we describe a very general sampling framework named freeform sampling. Freeform sampling handles three kinds of problems: sample reduction, minimum sampling rate to meet an error requirement, and minimization of reconstruction error given a fixed number of samples. When the to-be-reconstructed function values are unknown, freeform sampling becomes active sampling. Algorithms of active sampling are developed for light field and show better results than the traditional uniform sampling approach. Third, we present a self-reconfigurable camera array that we developed, which features a very efficient algorithm for real-time rendering andthe ability of automatically reconfiguring the cameras to improve the rendering quality. Both are based on active sampling. Our camera array is able to render dynamic scenes interactively at high quality. To the best of our knowledge, it is the first camera array that can reconfigure the camera positions automatically.

Autorentext

Cha Zhang is a Researcher in the Communication and Collaboration Systems Group at Microsoft Research (Redmond, WA). He received the B.S. and M.S. degrees from Tsinghua University, Beijing, China in 1998 and 2000, respectively, both in Electronic Engineering, and the Ph.D. degree in Electrical and Computer Engineering from Carnegie Mellon University, in 2004. His current research focuses on applying various machine learning and computer graphics/computer vision techniques to multimedia applications, in particular, multimedia teleconferencing. During his graduate studies at CMU, he worked on various multimedia related projects including sampling and compression of image-based rendering data, 3D model database retrieval and active learning for database annotation, peer-to-peer networking, etc. Dr. Zhang has published more than 40 technical papers and holds 10+ U.S. patents. He won the best paper award at ICME 2007, the top 10% award at MMSP 2009, and the best student paper award at ICME 2010. He co-authored a book titled Light Field Sampling, published by Morgan and Claypool in 2006. Dr. Zhang is a Senior Member of IEEE. He was the Publicity Chair for International Packet Video Workshop in 2002, the Program Co-Chair for the first Immersive Telecommunication Conference (IMMERSCOM) in 2007, the Steering Committee Co-Chair and Publicity Chair for IMMERSCOM 2009, the Program Co-Chair for the ACM Workshop on Media Data Integration (in conjunction with ACM Multimedia 2009), and the Poster&Demo Chair for ICME 2011. He served as TPC members for many conferences including ACM Multimedia, CVPR, ICCV, ECCV, MMSP, ICME, ICPR, ICWL, etc. He served as an Associate Editor for Journal of Distance Education Technologies, IPSJ Transactions on Computer Vision and Applications, and ICST Transactions on Immersive Telecommunications. He was a guest editor for Advances in Multimedia, Special Issue on Multimedia Immersive Technologies and Networking. Long Quan is a Professor of the Department of Computer Science and Engineering at the Hong Kong University of Science and Technology. He received his Ph.D. in Computer Science from INPL, France, in 1989. Before moving back to Hong Kong in 2001, he has been a French CNRS senior research scientist at INRIA in Grenoble. His research interests are focused on 3D reconstruction, structure from motion, vision geometry, and image-based modeling. He has served as an Associate Editor of PAMI (IEEE Transactions on Pattern Analysis and Machine Intelligence) and as a Regional Editor of IVC (Image and Vision Computing Journal). He is currently on the editorial board of IJCV (the International Journal of Computer Vision), ELCVIA (the Electronic Letters on Computer Vision and Image Analysis), MVA (Machine Vision and Applications) and Foundations and Trends in Computer Graphics and Vision. He has served as area chair for ICCV (International Conference on Computer Vision), ECCV (European Conference on Computer Vision), and CVPR (IEEE Computer Vision and Pattern Recognition) and ICPR (IAPR International Conference on Pattern Recognition). He was a Program Chair of ICPR 2006 Computer Vision and Image Analysis, and is a General Chair of ICCV 2011 in Barcelona.

Inhalt
The Light Field.- Light Field Spectral Analysis.- Light Field Uniform Sampling.- The Freeform Sampling Framework.- Light Field Active Sampling.- The Self-Reconfigurable Camera Array.- Conclusions and Future Work.