The emerging cloud gaming service has been growing rapidly, but not yet able to reach mobile customers due to many limitations, such as bandwidth and latency. In this talk, I will first provide an overview on 3D cloud gaming and survey existing cloud gaming solutions. This is followed by an introduction of the real-time video coding techniques used in OnLive, which is a well-known proprietary cloud gaming service. Then I will introduce a 3D image warping assisted real-time video coding method that can potentially meet all the requirements of mobile cloud gaming. The proposed video encoder selects a set of key frames in the video sequence, uses the 3D image warping algorithm to interpolate other non-key frames, and encodes the key frames and the residues frames with an H.264/AVC encoder. Our approach is novel in taking advantage of the run-time graphics rendering contexts (rendering viewpoint, pixel depth, camera motion, etc.) from the 3D game engine to enhance the performance of video encoding for the cloud gaming service. The experiments indicate that our proposed video encoder has the potential to beat the state-of-art x264 encoder
in the scenario of real-time cloud gaming. For example, by implementing the proposed method in a 3D tank battle game, we experimentally show that more than 2 dB quality improvement is possible.
Cheng-Hsin Hsu received the Ph.D. degree from Simon Fraser University, Canada in 2009, the M.Eng. degree from University of Maryland, College Park in 2003, and the M.Sc. and B.Sc. degrees from National Chung-Cheng University, Taiwan in 2000 and 1996, respectively. He is an Assistant Professor in Department of Computer Science at National Tsing Hua University, Taiwan. He was a Senior Research Scientist at Deutsche Telekom R&D Lab USA, Los Altos, CA between 2009 and 2011, and he was a Software Engineer with Motorola Inc. and Lucent Technologies before doing his PhD. His research interests are in the area of multimedia networking and distributed systems. He has published more than 45 papers in leading journals, conferences, and workshops. He and his colleagues at Simon Fraser University developed a mobile TV testbed, which won the Best
Technical Demo Award in the ACM multimedia 2008 Conference. He is on the Review Broad Committee of IEEE Technical Committee on Multimedia Communications (MMTC) and the Preservation Committee of ACM Special Interest Group on Multimedia (SIGMM). He served as the TPC Co-chair of the ACM Mobile Video Delivery Workshop (MoViD'11) and the Proceeding and Web Chair of the ACM International Workshop on Network and Operating Systems Support for Digital Audio and Video (NOSSDAV'11). He was on the technical program committees of several well-known
conferences in his research areas, including ACM Multimedia Conference (Multimedia), ACM International Workshop on Network and Operating Systems Support for Digital Audio and Video (NOSSDAV), and IEEE International Conference on Multimedia and Expo (ICME).